GCC

Simplifying Function Tracing for the Modern GCC

Zack Brown — Fri, 26 Jul 2019 11:30:00 +0000

Steven Rostedt wanted to do a little housekeeping, specifically with the function tracing code used in debugging the kernel. Up until then, the kernel could enable function tracing using either GCC's -pg flag or a combination of -pg and -mfentry. In each case, GCC would create a special routine that would execute at the start of each function, so the kernel could track calls to all functions. With just -pg, GCC would create a call to mcount() in all C functions, although with -pg coupled with -mfentry, it would create a call to fentry().

Steven pointed out that using -mfentry was generally regarded as superior, so much so that the kernel build system always would choose it over the mcount() alternative by testing GCC at compile time to see if it actually supported that command-line argument.

This is all very normal. Since any user might have any version of a given piece of software in the toolchain, or a variety of different CPUs and so on, each with different capabilities, the kernel build system runs many tests to identify the best available features that the kernel will be able to rely on.

But in this case, Steven noticed that for Linux version 4.19, Linus Torvalds had agreed to bump the minimum supported GCC version to 4.6. Coincidentally, as Steven now pointed out, GCC version 4.6 was the first to support the -mfentry argument. And, this was his point—all supported versions of GCC now supported the better function tracing option, and so there was no need for the kernel build system to cling to the mcount() implementation at all.

Steven posted a patch to rip it out by the roots.

Peter Zijlstra gave his support for this plan, as did Jiri Kosina. And, Jiri in particular spat upon the face of the mcount() solution.

Linus also liked Steven's patch, and he pointed out that with mcount() out of the picture, there were several more areas in the kernel that had existed simply to help choose between mcount() and fentry(), and that those now also could be removed. But Steven replied that, although yes this should be done, he still wanted to do split it up into a separate patch, for cleanliness' sake.

Go to Full Article

Crazy Compiler Optimizations

Zack Brown — Thu, 23 May 2019 11:30:00 +0000

by Zack Brown

Kernel development is always strange. Andrea Parri recently posted a patch to change the order of memory reads during multithreaded operation, such that if one read depended upon the next, the second could not actually occur before the first.

The problem with this was that the bug never could actually occur, and the fix made the kernel's behavior less intuitive for developers. Peter Zijlstra, in particular, voted nay to this patch, saying it was impossible to construct a physical system capable of triggering the bug in question.

And although Andrea agreed with this, he still felt the bug was worth fixing, if only for its theoretical value. Andrea figured, a bug is a bug is a bug, and they should be fixed. But Peter objected to having the kernel do extra work to handle conditions that could never arise. He said, "what I do object to is a model that's weaker than any possible sane hardware."

Will Deacon sided with Peter on this point, saying that the underlying hardware behaved a certain way, and the kernel's current behavior mirrored that way. He remarked, "the majority of developers are writing code with the underlying hardware in mind and so allowing behaviours in the memory model which are counter to how a real machine operates is likely to make things more confusing, rather than simplifying them!"

Still, there were some developers who supported Andrea's patch. Alan Stern, in particular, felt that it made sense to fix bugs when they were found, but that it also made sense to include a comment in the code, explaining the default behavior and the rationale behind the fix, even while acknowledging the bug never could be triggered.

But, Andrea wasn't interested in forcing his patch through the outstretched hands of objecting developers. He was happy enough to back down, having made his point.

It was actually Paul McKenney, who had initially favored Andrea's patch and had considered sending it up to Linus Torvalds for inclusion in the kernel, who identified some of the deeper and more disturbing issues surrounding this whole debate. Apparently, it cuts to the core of the way kernel code is actually compiled into machine language. Paul said:

We had some debates about this sort of thing at the C++ Standards Committee meeting last week.

Pointer provenance and concurrent algorithms, though for once not affecting RCU! We might actually be on the road to a fix that preserves the relevant optimizations while still allowing most (if not all) existing concurrent C/C++ code to continue working correctly. (The current thought is that loads and stores involving inline assembly, C/C++ atomics, or volatile get their provenance stripped. There may need to be some other mechanisms for plain C-language loads and stores in some cases as well.)

Go to Full Article

Moving Compiler Dependency Checks to Kconfig

Zack Brown — Wed, 19 Sep 2018 12:00:00 +0000

by Zack Brown

The Linux kernel config system, Kconfig, uses a macro language very similar to the make build tool's macro language. There are a few differences, however. And of course, make is designed as a general-purpose build tool while Kconfig is Linux-kernel-specific. But, why would the kernel developers create a whole new macro language so closely resembling that of an existing general-purpose tool?

One reason became clear recently when Linus Torvalds asked developers to add an entirely new system of dependency checks to the Kconfig language, specifically testing the capabilities of the GCC compiler.

It's actually an important issue. The Linux kernel wants to support as many versions of GCC as possible—so long as doing so would not require too much insanity in the kernel code itself—but different versions of GCC support different features. The GCC developers always are tweaking and adjusting, and GCC releases also sometimes have bugs that need to be worked around. Some Linux kernel features can only be built using one version of the compiler or another. And, some features build better or faster if they can take advantage of various GCC features that exist only in certain versions.

Up until this year, the kernel build system has had to check all those compiler features by hand, using many hacky methods. The art of probing a tool to find out if it supports a given feature dates back decades and is filled with insanity. Imagine giving a command that you know will fail, but giving it anyway because the specific manner of failure will tell you what you need to know for a future command to work. Now imagine hundreds of hacks like that in the Linux kernel build system.

Part of the problem with having those hacky checks in the build system is that you find out about them only during the build—not during configuration. But since some kernel features require certain GCC versions, the proper place to learn about the GCC version is at config time. If the user's compiler doesn't support a given feature, there's no reason to show that feature in the config system. It should just silently not exist.

Linus requested that developers migrate those checks into the Kconfig system and regularize them into the macro language itself. This way, kernel features with particular GCC dependencies could identify those dependencies and then show up or not show up at config time, according to whether those dependencies had been met.

That's the reason simply using make wouldn't work. The config language had to represent the results of all those ugly hacks in a friendly way that developers could make use of.

Go to Full Article

Bug Hunting Inlined Code

Zack Brown — Wed, 05 Sep 2018 13:08:08 +0000

by Zack Brown

The Linux kernel has various debugging tools. One is the kernel function tracer, which traces function calls, looking for bad memory allocations and other problems.

Changbin Du from Intel recently posted some code to increase the range of the function tracer by increasing the number of function calls that were actually compiled into the kernel. Not all function calls are ever actually compiled—some are "inlined", a C feature that allows the function code to be copied to the location that calls it, thus letting it run faster. The downside is that the compiled binary grows by the number of copies of that function it has to store.

But, not all inlined functions are specifically intended by the developers. The GNU C Compiler (GCC) also will use its own algorithms to decide to inline a wide array of functions. Whenever it does this in the Linux kernel, the function tracer has nothing to trace.

Changbin's code still would allow functions to be inlined, but only if they explicitly used the inline keyword of the C language. All other inlining done by GCC itself would be prevented. This would produce less efficient code, so Changbin's code never would be used in production kernel builds. But on the other hand, it would produce code that could be far more thoroughly examined by the function tracer, so Changbin's code would be quite useful for kernel developers.

As soon as he posted the patches, bug reports popped up all over the kernel in functions that GCC had been silently inlining. As a result, absolutely nobody had any objections to this particular patch.

There were, however, some odd false positives produced by the function tracer, claiming that it had found bugs that didn't actually exist. This gave a few kernel developers a slight pause, and they briefly debated how to eliminate those false positives, until they realized it didn't really matter. They reasoned that the false positives probably indicated a problem with GCC, so the GCC people would want to be able to see those false positives rather than have them hidden away behind workarounds.

That particular question—what is a kernel issue versus a GCC issue—is potentially explosive. It didn't lead anywhere this time, but in the past, it has led to bitter warfare between the kernel people and the GCC people. One such war was over GCC's failure to support Pentium processors and led to a group of developers forking GCC development into a competing project, called egcs. The fork was very successful, and it began to be used in mainstream Linux distributions instead of GCC. Ultimately, the conflict between the two branches was resolved only after the egcs code was merged into the GCC main branch, and future GCC development was handed over to the egcs team of developers in 1999.

Go to Full Article

Minimum GCC Version Likely to Jump from 3.2 to 4.8

Zack Brown — Wed, 11 Jul 2018 11:45:00 +0000

by Zack Brown

The question of the earliest GCC compiler version to support for building the Linux kernel comes up periodically. The ideal would be for Linux to compile under all GCC versions, because you never know what kind of system someone is running. Maybe their company's security team has to approve all software upgrades for their highly sensitive devices, and GCC is low on that list. Maybe they need to save as much space as possible, and recent versions of GCC are too big. There are all sorts of reasons why someone might be stuck with old software. But, they may need the latest Linux kernel because it's the foundation of their entire product, so they're stuck trying to compile it with an old compiler.

However, Linux can't really support every single GCC version. Sometimes the GCC people and the kernel people have disagreed on the manner in which GCC should produce code. Sometimes this means that the kernel really doesn't compile well on a particular version of GCC. So, there are the occasional project wars emerging from those conflicts. The GCC people will say the compiler is doing the best thing possible, and the kernel people will say the compiler is messing up their code. Sometimes the GCC people change the behavior in a later release, but that still leaves a particular GCC version that makes bad Linux code.

So, the kernel people will decide programmatically to exclude a particular version of GCC from being used to compile the kernel. Any attempt to use that compiler on kernel code will produce an error.

But, sometimes the GCC people will add a new language feature that is so useful, the kernel will people decide to rely heavily on it in their source code. In that case, there may be a period of time where the kernel people maintain one branch of code for the newer, better compiler, and a separate, less-fast or more-complex branch of code for the older, worse compiler. In that case, the kernel people—or really Linus Torvalds—eventually may decide to stop supporting compilers older than a certain version, so they can rip out all those less-fast and more-complex branches of code.

For similar reasons, it's also just an easier maintenance task for the kernel folks to drop support for older compilers; so this is something they would always prefer to do, if possible.

But, it's a big decision, typically weighed against the estimated number of users that are unable to upgrade their compilers. Linus really does not want even one regular (that is, non-kernel-developer) user to be unable to build Linux because of this sort of decision. He's willing to let the kernel carry a lot of fairly dead and/or hard-to-maintain code in order to keep that from happening.

Go to Full Article