With Nix, bitcoin projects can accelerate contributions and deployments

This is an opinion editorial by Nifty, a developer educator who runs the Base58 Bitcoin Protocol Academy and organizer of the upcoming, nix-focused Bitcoin++ conference.

When you want to install a new app on your computer, where do you get it?

Deciding which sources to use is a problem that’s surprisingly difficult to solve, but it’s a problem that every computing platform grapples with.

Prior to the wide spread of the Internet, new software “packages” were distributed on hard media such as CDs or USB sticks. For example, if you want to get the latest version of your computer’s operating system (OS) or the latest version of Bitcoin Core, you will need to buy it physically at a store or have it mailed to you. You’ll then load the disc or USB stick into your computer, and make a copy of the new software onto your local disk.

As a developer writing code, you must physically hand out copies of your program to friends and family. Or you’ll work for a large corporation that has relationships with companies and computer stores that will figure out how to distribute the software you write to their clients and customers.

With the advent of always-connected computers, the landscape of software distribution changed forever. Now, instead of physically carrying a copy of Bits in a program, you can easily download a copy of it from the Internet.

This made it very easy for new developers to write apps that anyone could download. However, the problem now lies in answering the question: where on the Internet should users download new software?

Hacker’s App Store

The way the mobile phone ecosystem solves the problem of new application distribution is through a program called the “App Store” (or “Google Play Store” on Android) on your phone.

Developers upload the latest copy of their apps to Apple’s or Google’s servers. Those companies then send a silent notification to all phones that have the app installed, and the app gets updated in the background on the users’ phones, without the users’ involvement.

If you’re a user and you want to add a new app to your phone, in most cases you’ll be able to find it in the App Store, and download it from there.

But not all software is available as a mobile phone app. Some software projects are desktop apps or tools that developers use to write new pieces of software. These applications also need a distribution channel. On desktop operating systems, we call “App Store” software by the term “package manager”. They manage packages of software that you download onto your computer.

Modern package managers have a central server, sometimes called a repository, that can keep track of the latest versions of the app and client-side applications that run on the package user’s computer.

If you’re a developer, you’ve probably encountered “homebrew” or “apt-get install” somewhere in your life. “homebrew” and “apt” are package managers.

The way package managers work, the client-side application will check in with the central repository occasionally to see if there are any new updates for any of its currently installed packages. Or, if you have the right OS and the right permissions set, your computer might automatically run these checks, download and install the latest version of the app.

(As an aside, you can add other sources of software packages to your local package-manager client. If a developer self-hosts their apps instead of uploading them to a central repository, you can add their self-hosted website to it.) You’ll need to add your list of places to look for updates in your package-manager client app.)

What if we could reproduce everything?

cipher A package manager whose goal is to bring reproducible builds to all the packages it helps distribute.

Nix solves software distribution problems by keeping detailed track of all inputs to a build, uniquely identifying each input, and using them in the same way as the original builder. This ensures that it is able to create the same thing every time and on any machine.

Nix hashes every input and output of a build; This makes it easy to identify when an input has changed. When the inputs change in a build, the outputs are guaranteed to change as well.

A major problem with reproducible builds is that different users will have different versions of the inputs a program needs to be built with. Each new program you create on your machine may require a different version of its input to reproduce.

Nix fixes this by keeping track of builds by hash lists and only using inputs that match a list of expected hashes. The tradeoff is that nix uses a lot more disk space and can often take a bit longer to download, as it will need to download the correct inputs to the binary build process if you don’t already have them on your computer.

I love to joke that the Knicks were only possible thanks to Moore’s LawWhich means we now have enough extra disk space that we don’t have to worry about multiple copies of the same program, just in slightly different versions.

Therefore, Nix makes it possible to more easily distribute reproducible builds of your software, meaning a user can reliably rebuild your software on their own machine and just get it working, no questions asked. .

As a small aside, note that “reproducible” for nix builds means “if a package works on one machine, it will also work on another machine.” This is not the same as the reproducibility that the Bitcoin Core developers try to guarantee for official Gitcoin binaries, which is a different process to check that they are bit-for-bit identical.

nix and bitcoin

Distributing open-source software in a way that is easy for end users to run, and can be reproduced on any system, is a very useful and important feature. This makes it easy for anyone to start using the project and also makes it easy for new developers to contribute back to the project.

A few years ago, a group of intrepid developers launched a repository of the most common bitcoin software projects that have been packaged as nix packages and modules.

Packages make it very easy for anyone using the Nix package manager to get started installing and running bitcoind node on almost any architecture. You don’t have to worry about installing the correct dependencies before running the build command, and the project will build from source on your machine. This is pretty ideal in terms of being able to instantly download, build and run a bitcoin node.

nix-bitcoin repository Nix provides “modules”, which are configurations for software applications to run on. nixOS, The nix-bitcoin module allows anyone running NixOS to reproduce builds of Bitcoin Core and other bitcoin open-source projects, and get them running quickly on their own machines with a minimum of hassle.

In an interview for this article, Jonas Nicks, one of the lead maintainer of the nix-bitcoin projectThe project had the following to say about using it:

“The main advantage that nix-bitcoin gets over Nix/NixOS is declarative configuration management. Configuring your system using the Nix programming language allows for abstraction, so that you can reuse and test individual components using your own logic.” modularization. This allows to manage the complexity, which inevitably occurs when building a system, and helps to build a perfect and secure system. For example, many of those provided by nix-bitcoin Maintaining security features would be very annoying if we didn’t have a module system. Reproducibility in NixOS also helps a lot with updates, because we know what’s going on on our users’ systems. Finally, with NixOS Errors can also occur at evaluation time rather than at runtime.”

Bitcoin Core is not the only project that has been packaged for distribution on Nix.

It also has Lightning Core, BTCPay Server and many other projects. For the most part, these projects have been linked and bundled for distribution on Nix by fans of the Nix package-management system, not by the developers who wrote the software. For a more complete list of available packages, see the “Modules” list nix-bitcoin GitHub repository,

Why not nix everywhere?

If Nix is ​​such a huge change to the way bitcoin projects are built and shipped, why isn’t everyone using it? The truth is that nix is ​​a big project that is a bit difficult to get involved in. The learning curve for using Nix is ​​quite steep, and the paradigm of running Nix as a package manager can be quite different than what most people are used to.

NixOS started as a research project Elko Dolstra in 2003And since then slowly building up followers and user base.

bitcoin++ berlin: expanding the use of nix in bitcoin

by organizing upcoming bitcoin++ conference In Berlin, we aim to change that. bitcoin++ is a developer-focused conference series focused on a variety of topics in the bitcoin developer sphere. Originally set to begin in Austin, Texas in 2022, the most recent conference in late April 2023 focused on Layer 2 technologies in bitcoin, such as Arc, the Lightning Network.

and Validity Rollup.

The upcoming Berlin conference, planned for October 6 and October 7, 2023, aims to be a “nix-piling” event for the bitcoin developer ecosystem. The main maintainers of the Nix-Bitcoin project (including Nick, Pavol Rusnak and Eric Sirian) will be present at Fort Nix

name a few) The two-day event will be filled with practical workshops and lectures

Designed to teach developers of the entire ecosystem how to use nix for packages

better bundle their projects and their development dependencies so that the new

Contributors can start coding and contributing faster, as well as demonstrate how fast and easy it is to deploy software using modules on NixOS.

Additional Articles References

1. reproducibility with nix
2. Interview with “nixbitcoindev”
3. where to start
4. Why Nix-Bitcoin for Bitcoiners

This is a guest post by niftynee. The opinions expressed are solely his own and do not necessarily reflect the opinions of BTC Inc. or Bitcoin Magazine.