git

Documenting Proper Git Usage

Zack Brown — Wed, 07 Aug 2019 22:30:00 +0000

Jonathan Corbet wrote a document for inclusion in the kernel tree, describing best practices for merging and rebasing git-based kernel repositories. As he put it, it represented workflows that were actually in current use, and it was a living document that hopefully would be added to and corrected over time.

The inspiration for the document came from noticing how frequently Linus Torvalds was unhappy with how other people—typically subsystem maintainers—handled their git trees.

It's interesting to note that before Linus wrote the git tool, branching and merging was virtually unheard of in the Open Source world. In CVS, it was a nightmare horror of leechcraft and broken magic. Other tools were not much better. One of the primary motivations behind git—aside from blazing speed—was, in fact, to make branching and merging trivial operations—and so they have become.

One of the offshoots of branching and merging, Jonathan wrote, was rebasing—altering the patch history of a local repository. The benefits of rebasing are fantastic. They can make a repository history cleaner and clearer, which in turn can make it easier to track down the patches that introduced a given bug. So rebasing has a direct value to the development process.

On the other hand, used poorly, rebasing can make a big mess. For example, suppose you rebase a repository that has already been merged with another, and then merge them again—insane soul death.

So Jonathan explained some good rules of thumb. Never rebase a repository that's already been shared. Never rebase patches that come from someone else's repository. And in general, simply never rebase—unless there's a genuine reason.

Since rebasing changes the history of patches, it relies on a new "base" version, from which the later patches diverge. Jonathan recommended choosing a base version that was generally thought to be more stable rather than less—a new version or a release candidate, for example, rather than just an arbitrary patch during regular development.

Jonathan also recommended, for any rebase, treating all the rebased patches as new code, and testing them thoroughly, even if they had been tested already prior to the rebase.

"If", he said, "rebasing is limited to private trees, commits are based on a well-known starting point, and they are well tested, the potential for trouble is low."

Moving on to merging, Jonathan pointed out that nearly 9% of all kernel commits were merges. There were more than 1,000 merge requests in the 5.1 development cycle alone.

Go to Full Article

Signing Git Commits

Kyle Rankin — Thu, 16 May 2019 12:00:00 +0000

by Kyle Rankin

Protect your code commits from malicious changes by GPG-signing them.

Often when people talk about GPG, they focus on encryption—GPG's ability to protect a file or message so that only someone who has the appropriate private key can read it. Yet, one of the most important functions GPG offers is signing. Where encryption protects a file or message so that only the intended recipient can decrypt and read it, GPG signing proves that the message was sent by the sender (whomever has control over the private key used to sign) and has not been altered in any way from what the sender wrote.

Without GPG signing, you could receive encrypted email that only you could open, but you wouldn't be able to prove that it was from the sender. But, GPG signing has applications far beyond email. If you use a modern Linux distribution, it uses GPG signatures on all of its packages, so you can be sure that any software you install from the distribution hasn't been altered to add malicious code after it was packaged. Some distributions even GPG-sign their ISO install files as a stronger form of MD5sum or SHA256sum to verify not only that the large ISO downloaded correctly (MD5 or SHA256 can do that), but also that the particular ISO you are downloading from some random mirror is the same ISO that the distribution created. A mirror could change the file and generate new MD5sums, and you may not notice, but it couldn't generate valid GPG signatures, as that would require access to the distribution's signing key.

Why Sign Git Commits

As useful as signing packages and ISOs is, an even more important use of GPG signing is in signing Git commits. When you sign a Git commit, you can prove that the code you submitted came from you and wasn't altered while you were transferring it. You also can prove that you submitted the code and not someone else.

Being able to prove who wrote a snippet of code isn't so you know who to blame for bugs so the person can't squirm out of it. Signing Git commits is important because in this age of malicious code and back doors, it helps protect you from an attacker who might otherwise inject malicious code into your codebase. It also helps discourage untrustworthy developers from adding their own back doors to the code, because once it's discovered, the bad code will be traced to them.

How to Sign Git Commits

The simplest way to sign Git commits is by adding the -S option to the git commit command. First, figure out your GPG key ID with:

Go to Full Article

Take Your Git In-House

John S. Tonello — Thu, 27 Sep 2018 12:56:39 +0000

by John S. Tonello

If you're wary of the Microsoft takeover of GitHub, or if you've been looking for a way to ween yourself off free public repositories, or if you want to ramp up your DevOps efforts, now's a good time to look at installing and running GitLab yourself. It's not as difficult as you might think, and the free, open-source GitLab CE version provides a lot of flexibility to start from scratch, migrate or graduate to more full-fledged versions.

In today's software business, getting solid code out the door fast is a must, and practices to make that easier are part of any organization's DevOps toolset. Git has risen to the top of the heap of version control tools, because it's simple, fast and makes collaboration easy.

For developers, tools like Git ensure that their code isn't just backed up and made available to others, but nearly guarantees that it can be incorporated into a wide variety of third-party development tools—from Jenkins to Visual Studio—that make continuous integration and continuous delivery (CI/CD) possible. Orchestration, automation and deployment tools easily integrate with Git as well, which means code developed on any laptop or workstation anywhere can be merged, branched and integrated into deployed software. That's why version control repositories are the future of software development and DevOps, no matter how big or small you are, and no matter whether you're building monolithic apps or containerized ones.

Getting Started with Git

Git works by taking snapshots of code on every commit, so every version of contributed code is always available. That means it's easy to roll back changes or look over different contributors' work.

If you're working in an environment that uses Git, you can do your work even when you're offline. Everything is saved in a project structure on your workstation, just as it is in the remote Git repository, and when you're next online, your commits and pushes update the master (or other) code branch quickly and easily.

Most Git users (even newbies) use the Git command-line tools to clone, commit and push changes, because it's easy, and for nearly 28-million developers, GitHub has become the de facto remote Git-based repository for their work. In fact, GitHub has moved beyond being just a code repository to become a multifaceted code community featuring 85-million projects. That's a lot of code.

Go to Full Article

Git Quick Start Guide

Patrick Whelan — Tue, 14 Aug 2018 12:00:00 +0000

by Patrick Whelan

Ditch USBs and start using real version control, and if you follow this guide, you can start using git in 30 minutes!

If you have any experience with programming or just altering config files, I'm sure you've been dumbstruck by how one change you've made along the line affects the whole project. Identifying and isolating the problem without a version control system is often time- and energy-intensive, involving retracing your steps and checking all changes made before the unwanted behavior first occurred. A version control system is designed explicitly to make that process easier and provide readable comparisons between versions of text.

Another great feature that distributed version control systems such as git provide is the power of lateral movement. Traditionally, a team of programmers would implement features linearly. This meant pulling the code from the trusted source (server) and developing a section before pushing the altered version back upstream to the server. With distributed systems, every computer maintains a full repository, which means each programmer has a full history of additions, deletions and contributors as well as the ability to roll back to a previous version or break away from the trusted repository and fork the development tree (which I discuss later).

Quick Start Guide

The great thing about git is there's so little you need to know! Without further ado, let's begin with the most important commands.

First, I'm working with a previous project of mine located here:


[user@lj src]$ pwd
/home/lj/projects/java/spaceInvaders/src

To create a local repository, simply run:


[user@lj src]$ git init
Initialized empty Git repository in
 ↪/home/lj/projects/java/spaceInvaders/src/.git/

To add all source files recursively to git's index, run:


[user@lj src]$ git add .

To push these indexed files to the local repository, run:


[user@lj src]$ git commit

You'll see a screen containing information about the commit, which allows you to leave a description of the commit:

Go to Full Article

A Git Origin Story

Zack Brown — Fri, 27 Jul 2018 11:45:00 +0000

by Zack Brown

A look at Linux kernel developers' various revision control solutions through the years, Linus Torvalds' decision to use BitKeeper and the controversy that followed, and how Git came to be created.

Originally, Linus Torvalds used no revision control at all. Kernel contributors would post their patches to the Usenet group, and later to the mailing list, and Linus would apply them to his own source tree. Eventually, Linus would put out a new release of the whole tree, with no division between any of the patches. The only way to examine the history of his process was as a giant diff between two full releases.

This was not because there were no open-source revision control systems available. CVS had been around since the 1980s, and it was still the most popular system around. At its core, it would allow contributors to submit patches to a central repository and examine the history of patches going into that repository.

There were many complaints about CVS though. One was that it tracked changes on a per-file basis and didn't recognize a larger patch as a single revision, which made it hard to interpret the past contributions of other developers. There also were some hard-to-fix bugs, like race conditions when two conflicting patches were submitted at the same time.

Linus didn't like CVS, partly for the same reasons voiced by others and partly for his own reasons that would become clear only later. He also didn't like Subversion, an open-source project that emerged around the start of the 2000s and had the specific goal of addressing the bugs and misfeatures in CVS.

Many Linux kernel developers were unhappy with the lack of proper revision control, so there always was a certain amount of community pressure for Linus to choose something from one of the available options. Then, in 2002, he did. To everyone's shock and dismay, Linus chose BitKeeper, a closed-source commercial system developed by the BitMover company, run by Larry McVoy.

The Linux kernel was the most important open-source project in history, and Linus himself was the person who first discovered the techniques of open-source development that had eluded the GNU project, and that would be imitated by open-source projects for decades to come, right up to the present day. What was Linus thinking? How could he betray his community and the Open Source world like this? Those were the thoughts of many when Linus first started using BitKeeper for kernel development.

Go to Full Article

Building a Bare-Bones Git Environment

Andy Carlson — Tue, 24 Jul 2018 12:00:00 +0000

by Andy Carlson

How to migrate repositories from GitHub, configure the software and get started with hosting Git repositories on your own Linux server.

With the recent news of Microsoft's acquisition of GitHub, many people have chosen to research other code-hosting options. Self-hosted solutions like GitLabs offer a polished UI, similar in functionality to GitHub but one that requires reasonably well-powered hardware and provides many features that casual Git users won't necessarily find useful.

For those who want a simpler solution, it's possible to host Git repositories locally on a Linux server using a few basic pieces of software that require minimal system resources and provide basic Git functionality including web accessibility and HTTP/SSH cloning.

In this article, I show how to migrate repositories from GitHub, configure the necessary software and perform some basic operations.

Migrating Repositories

The first step in migrating away from GitHub is to relocate your repositories to the server where they'll be hosted. Since Git is a distributed version control system, a cloned copy of a repository contains all information necessary for running the entire repository. As such, the repositories can be cloned from GitHub to your server, and all the repository data, including commit logs, will be retained. If you have a large number of repositories this could be a time-consuming process. To ease this process, here's a bash function to return the URLs for all repositories hosted by a specific GitHub user:


genrepos() {
    if [ -z "$1" ]; then
        echo "usage: genrepos "
    else
        repourl="https://github.com/$1?tab=repositories"
        while [ -n "$repourl" ]; do
            curl -s "$repourl" | awk '/href.*codeRepository/
 ↪{print gensub(/^.*href="\/(.*)\/(.*)".*$/,
↪"https://github.com/\\1/\\2.git","g",$0); }'
            export repourl=$(curl -s "$repourl" | grep'>Previous<.
↪*href.*>Next<' | grep -v 'disabled">Next' | sed
↪'s/^.*href="//g;s/".*$//g;s/^/https:\/\/github.com/g')
        done
    fi
}

This function accepts a single argument of the GitHub user name. If the output of this command is piped into a while loop to read each line, each line can be fed into a git clone statement. The repositories will be cloned into the /opt/repos directory:

Go to Full Article

Opinion: GitHub vs GitLab

Matt Lee — Mon, 16 Jul 2018 11:30:00 +0000

by Matt Lee

Free software deserves free tools, not Microsoft-owned GitHub.

So, Microsoft bought GitHub, and many people are confused or worried. It's not a new phenomenon when any large company buys any smaller company, and people are right to be worried, although I argue that their timing is wrong. Like Microsoft, GitHub has made some useful contributions to free and open-source software, but let's not forget that GitHub's main product is proprietary software. And, it's not just some innocuous web service either; GitHub makes and sells a proprietary software package you can download and run on your own server called GitHub Enterprise (GHE).

Let's remember how we got here. BitMover made a tool called BitKeeper, a proprietary version control system that allowed free-of-charge licenses to free software projects. In 2002, the Linux kernel switched to using BitKeeper for its version control, although some notable developers made the noble choice to refuse to use the proprietary program. Many others did not, and for a number of years, kernel development was hampered by BitKeeper's restrictive noncommercial licenses.

In 2005, Andrew Tridgell, working at OSDL, developed a client that bypassed this restriction, and as a result, BitMover removed licenses to BitKeeper from all OSDL employees—including Linus Torvalds. Eventually, all non-commercial licenses were stopped, and new licenses included clauses preventing the development of alternative version control systems. As a result of this, two new projects were born: Mercurial and Git. Created in a few short weeks in 2005, Git quickly became the version control system for Linux development.

Proprietary version control tools aren't common in free software development, but proprietary collaboration websites have been around for some time. One of the earliest collaboration websites still around today is Sourceforge. Sourceforge was created in the late 1990s by VA Software, and the code behind the project was released in 2000.

Quickly this situation changed, and the project was shuttered and then became Sourceforge Enterprise Edition, a proprietary software package. The code that ran Sourceforge was forked into GNU Savannah (later Savane) and GForge, and it's still use today by both the GNU Project and CERN. When I last wrote about this problem, almost exactly ten years ago, Canonical's ambitious Launchpad service still was proprietary, something later remedied in 2009. Gitorious was created in 2010 and was for a number of years the Git hosting platform for the discerning free software developer, as the code for Gitorious was fully public and licensed under favorable terms for the new wave of AGPL-licensed projects that followed the FSF's Franklin Street Statement. Gitorious, also, is sadly no longer with us.

Go to Full Article

Git Your July 2018 Issue of Linux Journal: Now Available

Carlie Fairchild — Mon, 02 Jul 2018 17:30:00 +0000

by Carlie Fairchild

Along with Microsoft buying Github recently, we received hundreds of questions and comments about all things git. How does one install and run GitLab themselves? Should they? What's the difference between GitHub and GitLab? How can one migrate repositories from GitHub and host on their own Linux server? So with this July issue of Linux Journal, we take a Deep Dive in to... git. Enjoy!

Feature articles include:

A Git Origin Story by Zack Brown

A look at the Linux kernel developers' various revision control solutions through the years, Linus Torvalds' decision to use BitKeeper and the controversy that followed, and how Git came to be created.

Git Quick Start Guide by Patrick Whelan

Ditch USBs and start using real version control, and if you follow this guide, you can start using git in 30 minutes!

Building a Bare-Bones Git Environment by Andy Carlson

How to migrate repositories from GitHub, configure the software and get started with hosting Git repositories on your own Linux server.

Take Your Git In-House by John S. Tonello

Terrible Ideas in Git by Corey Quinn

This article was derived from a talk that GitHub Universe faithfully rejects every year. I can't understand why....

Opinion: GitHub vs GitLab by Matt Lee

Free software deserves free tools, not Microsoft-owned GitHub.

Other articles in this issue:

Encrypting NFSv4 with Stunnel TLS by Charles Fisher

NFS clients and servers push file traffic over clear-text connections in the default configuration, which is incompatible with sensitive data. TLS can wrap this traffic, finally bringing protocol security. Before you use your cloud provider's NFS tools, review all of your NFS usage and secure it where necessary.

Advertising 3.0 by Doc Searls

Road to RHCA—Preparation Meets Opportunity by Taz Brown

FOSS Project Spotlight: ONLYOFFICE, an Online Office Suite by Tatiana Kochedykova

At Rest Encryption by Kyle Rankin

Progress with Your Image by Kyle Rankin

FOSS Project Spotlight: Pydio Cells, an Enterprise-Focused File-Sharing Solution by Italo Vignoli

Atomic Modeling with GAMGI by Joey Bernard

News Briefs by Jill Franklin

Kyle Rankin's Hack and /: What Really IRCs Me: Slack

Reuven M. Lerner's At the Forge: Introducing Python 3.7's Dataclasses

Go to Full Article

Terrible Ideas in Git

Corey Quinn — Mon, 02 Jul 2018 13:50:25 +0000

by Corey Quinn

This article was derived from a talk that GitHub Universe faithfully rejects every year. I can't understand why....

For better or worse, git has become one of the Open Source community's more ubiquitous tools. It lets you manage code effectively. It helps engineers who are far apart collaborate with each other. At its heart, it's very simple, which is why the diagram in so many blog posts inevitably looks something like the one shown in Figure 1.

Figure 1. Git Model (Source: https://nvie.com)

The unfortunate truth that's rarely discussed in detail is that git has a dark side: it makes us feel dumb. I don't care who you are—we all hit a point wherein we shrug, give up and go scrambling for Stack Overflow (motto: "This thread has been closed as Off Topic") to figure out how best to get out of the terrible situations we've caused for ourselves. The only question is how far down the rabbit hole you can get before the madness overtakes you, and you begin raising goats for a living instead.

At its core, all git does is track changes to files and folders. git commit effectively takes a snapshot of the filesystem (as represented by the items added to the staging area) at a given point in time:


cquinn@1d732dc08938 ~/demo1 % git init
Initialized empty Git repository in /home/cquinn/demo1/.git/
cquinn@1d732dc08938(master|...) ~/demo1 % git add ubuntu.iso
cquinn@1d732dc08938(master|·1) ~/demo1 % git commit
 ↪-m "Initial commit"
[master (root-commit) b0d3bfb] Initial commit
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 ubuntu.iso
cquinn@1d732dc08938(master|✓) ~/demo1 % git rm --cached
 ↪ubuntu.iso
rm 'ubuntu.iso'
cquinn@1d732dc08938(master|·1✓) ~/demo1 % git
 ↪commit -m "There I fixed it"
[master 2d86934] There I fixed it
 1 file changed, 0 insertions(+), 0 deletions(-)
 delete mode 100644 ubuntu.iso
cquinn@1d732dc08938(master|...) ~/demo1 % du -hs .git
174M    .git

So if you do something foolish, such as committing large binaries, you can't just revert the commit—it's still going to live in your git repository. If you've pushed that thing elsewhere, you get to rewrite history forcibly, either with git-filter-branch or the bfg. Either way, it's extra work that's unpleasant to others who share your repository.

Fundamentally, all that git does is create a .git folder in the top level of the repository. This subdirectory contains files and folders that change over time. Wait, isn't there a tool for that?

Go to Full Article

Microsoft Buys GitHub: Three Weeks Later

Marcel Gagné — Tue, 26 Jun 2018 11:30:00 +0000

by Marcel Gagné

I heard that Microsoft would be buying GitHub just a couple days before it happened when Carlie Fairchild at Linux Journal told me about it. I replied to the news with a solid, “Get! Out!” Needless to say, I had my doubts. As someone who remembers all too well the “Embrace, extend and extinguish" days of Microsoft, the news of this latest embrace did, however briefly, bring back those old memories. When I was asked what I thought, I answered that the optics were bad.A lot of years have passed since, back in 2001, Steve Ballmer declared Linux to be a cancer. These days, Microsoft loves Linux. It says so right on its website. Two years ago, Steve Ballmer also proclaimed his love for Linux. In 2018, Microsoft has its own distribution that it uses in its Azure cloud. Microsoft includes several different flavors of Linux in its app store (the Windows Subsystem for Linux), all of which can be installed on Windows 10. Microsoft develops for Linux. Heck, Microsoft even contributes to the Linux kernel.

The reason I felt the optics were bad is that Microsoft has spent the last few years going out of its way to paint itself as a friend to Linux and open source. This, I thought, can only be seen as a bad move. Well, it’s been three weeks, more or less, since the the acquisition became official, to the tune of $7.5 billion US. What happened?

For starters, Jim Zemlin, the Executive Director of the Linux Foundation, praised the Microsoft acquisition of GifHub, suggesting that it could be a good thing for everyone. Former open-source star and the new CEO of GitHub, Nat Friedman, did an AMA on Reddit to reassure developers of open-source software that they had nothing to fear from the new landlords.

Linus Torvalds, the creator of the Linux kernel and git itself, didn’t comment directly, but he has been critical of GitHub in the past, so perhaps it doesn’t change anything for him. Besides, the kernel is primarily housed on a private git server, and GitHub is just a public mirror for the code.

That didn’t stop some number of open-source developers from swearing off GitHub and looking for other places to host their projects. GitLab, a minor competitor to GitHub, seemed poised to be the natural beneficiary of this move. In a Twitter post dated June 3rd, it cited that its GitHub to GitLab conversion rate was running at ten times normal.

Go to Full Article