Chapter 2. Cryptocurrency Fundamentals

Like many emerging technologies, cryptocurrencies have ushered in a new way of thinking—about finance, in this case. As a result, new ways of storing value are being considered. Cryptocurrency has some similarities to fiat money, stocks, and bonds, as well as to precious assets such as gold. But the methods of acquiring, transferring, and storing cryptocurrency, or crypto for short, are very different from other assets.

Note

Major differences are involved regarding security and sending/receiving value in cryptocurrencies. It is possible to lose funds in these systems without recourse.

Using cryptocurrency can be confusing as it introduces a large number of new terms that even people in traditional finance are often unfamiliar with. Although today there are a number of services that make a lot of the basic underpinnings of crypto easier for consumers, becoming familiar with at least the basics of how cryptocurrency systems work can be helpful when trying to understand how these new systems are different. This chapter will cover some basic terminology and processes that will set you up for understanding the rest of the book.

Note

Although the examples here will primarily use bitcoin, most of these concepts translate to other cryptocurrencies as well. Ethereum introduces some new ideas, which are covered starting in Chapter 4.

Public and Private Keys in Cryptocurrency Systems

The use of cryptography, or encryption, goes back thousands of years. Its primary use has been to safeguard information. Up until the twentieth century, all that was needed for rudimentary cryptography was a writing implement and something to write on, which could be used to create rudimentary codes. In modern times, the use of machinery and subsequently computers fueled encryption’s uses. For example, the success of the Allies in eventually breaking and deciphering German cryptography in World War II contributed to the end of that conflict.

Modern cryptography was used mostly by governments up until the 1970s. It was the invention of public key, or asymmetric, cryptography that allowed its use to become more widespread, along with the growing proliferation of computer networks. Public key cryptography allows anyone to easily encrypt a message using a receiver’s public key, which is available to anybody; the message can then be decrypted using the receiver’s private key, which (hopefully) only the recipient knows.

Cryptography has become crucial to major systems that underpin the internet. This includes the fundamentals underlying cryptocurrencies such as Bitcoin. For example, signing up for a hosted Bitcoin wallet generates a public key and a private key, and a Bitcoin address (we’ll talk more about wallets in “Custody: Who Holds the Keys”). The Bitcoin address is a translation of the public key and is the identity of the wallet where funds can be received and from which they can be sent to other addresses. This address can be shared with anybody for receiving and sending, a bit like a username or email address. The private key is kept secret and is used to unlock stored cryptocurrency, somewhat like how you use a password to access your bank account. Here’s an example of what they look like:

Private key Kyc9JCPPKNPrMUopkCc7ng9PU5Bp9SGsjVkh8Hpfx4tCr5LGXgBf
Public key 033b368bfccf5921f8a5a42b81b0f5ecdc66583fac8dc13bcf860cf31290964c64
Bitcoin address 19PacjCFSSt9guX4zZ3GPpXpDrvDNQ7DC4

As mentioned in the previous chapter, Bitcoin private keys are used to digitally sign transactions. That’s how the owner of a Bitcoin address proves to the Bitcoin network that they are the rightful owner of that address, and how they authorize a transaction. In the real world, a bank card user walks up to an ATM, inserts their card, and enters their personal identification number (PIN). That PIN is private—only the user is supposed to know it. For the ATM network, a valid PIN indicates that the ATM user has “signed” a transaction.

Bitcoin public keys are only used to generate a Bitcoin address. This address is essentially a compressed version of the public key, making it much easier to read. A Bitcoin address can be shared publicly with anyone, often when requesting that someone send bitcoin to that address.

A private key in Bitcoin is a random 256-bit number that is created by a random generator. Private keys are displayed in hexadecimal format. Once a user has a private key, it can be paired with a public key to unlock a Bitcoin address. Everything starts with the private key. From this, a user can generate a public key and Bitcoin address. As a reminder, Figure 2-1 illustrates the process for generating a Bitcoin address and which encryption algorithms are used at each step.

Figure 2-1. The process of finding a Bitcoin address from a private key

As you can see, the public key is generated by running the private key through an ECDSA secp256k1 function. The public key hash is generated by running the public key through the SHA256 and RIPEMD160 functions. The Bitcoin address is generated by first adding 00 to the public key hash and then running that value through a Base58Check function.

The UTXO Model

Bitcoin transactions follow a unique type of accounting called unspent transaction output (UTXO). A bitcoin transaction is essentially a list of inputs and a list of outputs. Each input identifies a Bitcoin address that is providing the funds, and an unspent transaction that address has received in the past. Similarly, each output represents the Bitcoin address receiving the funds and the amount that address receives. The difference between the input and the output is the transaction fee, which will be earned by the bitcoin miner. Each input also contains a digital signature, proving that the owner of that Bitcoin address authorizes that transaction. Figure 2-2 shows an example of a bitcoin transaction.

Figure 2-2. A sample transaction, and how it looks in a Bitcoin block explorer

In this example, there are four inputs. Two of the inputs come from the same address (1HXpg8D9AMGFVZ9FEU2tkZYvAZ8xBhVudo). However, these two inputs represent two different transactions that address has received in the past, one for 0.0027867 BTC and another for 0.0034977 BTC. The other inputs represent transactions that the addresses 14yPyVmGhNCSM9JgaabRZ8C3cT2RWEGd71 and 1MXDLBc2Tq2hnQ2x5qXTEPUen5xq9hDA39 have received in the past.

The inputs in this case total 0.0128 BTC, and the total amount output is 0.01145732 BTC. The difference between the two, 0.00134268 BTC, is paid to the miner who added the block the transaction is in to the blockchain (known as “mining” a block). The creator of the transaction pays this fee to incentivize the miner to include this transaction ahead of other transactions that are in the memory pool, where all valid transactions are waiting to be confirmed. We’ll talk more about all of these concepts later in the chapter, but first let’s take a look at how a bitcoin transaction is structured. Here’s an example of a raw transaction:

01000000017967a5185e907a25225574544c31f7b059c1a191d65b53dcc1554d339c4f9efc0100000
06a47304402206a2eb16b7b92051d0fa38c133e67684ed064effada1d7f925c842da401d4f2270220
1f196b10e6e4b4a9fff948e5c5d71ec5da53e90529c8dbd122bff2b1d21dc8a90121039b7bcd0824b
9a9164f7ba098408e63e5b7e3cf90835cceb19868f54f8961a825ffffffff014baf21000000000019
76a914db4d1141d0048b1ed15839d0b7a4c488cd368b0e88ac00000000

The fields are described in Table 2-1.

Table 2-1. Anatomy of a bitcoin transaction
Field Description
Version no. 4 bytes. Identifies which protocol version the node generating the transaction is using (currently 1).
Flag If the flag is present, showing a value of 0001, then the node is using Segregated Witness (SegWit), which removes signature information from the transaction.
In-counter The number of inputs.
List of inputs List of input data.
Out-counter The number of outputs.
List of outputs List of output data.
Witnesses If using SegWit, then this field shows a list of witnesses.
Lock time 4 bytes. If this field is not empty, it identifies the earliest time that the transaction can be added to the blockchain as determined by the network. This field can be represented as either a block height or a Unix-like timestamp.

Transactions

In cryptocurrency, transactions represent the movement of value from one address to another. Transactions that are published on the blockchain are said to be confirmed. Transfer of control of funds requires signing a transaction with a private key. The corresponding public key is then used by the recipient to verify the signature and validate the transaction. In most cryptocurrencies, users must also pay a small fee to the network in order for a transaction to go through. This fee, which usually goes to the miner who discovered the block in which the transaction is included, is to incentivize the miners publishing blocks to continue to secure the network. Online tools like the Bitcoin Fee Estimator & Calculator can help you predict how big a fee you should include with a transaction to get it recorded on the blockchain within the desired time frame (see Figure 2-3).

Figure 2-3. Estimating the fee to get a transaction included on the blockchain

Bitcoin transaction fees can vary depending on network capacity, how quickly confirmation is needed, and other factors. Because there is a limit on the number of transactions that can be recorded on a block—the current limit is 1 MB of data, or roughly 3,500 transactions per block—a higher fee may be required for greater urgency. There is essentially a competition in place for getting miners to confirm a transaction: higher fees mean faster confirmation. Figure 2-4 illustrates the series of events that occur in executing a bitcoin transaction.

Figure 2-4. Series of events involved in executing a bitcoin transaction—“block mined onto blockchain” refers to miners adding a new block to be confirmed by the network

The Merkle Root

The Merkle root is used to show a snapshot of the state of all the transactions in the current block, stored in just 256 bits. The name comes from computer scientist Ralph Merkle, who came up with Merkle trees, which are digital signature data structures. The Merkle root has a special purpose aside from capturing the transaction snapshot. When a node in the network wants to ensure it has the exact same list of transactions as every other node, it does not need to compare each transaction individually. Instead, it only needs to compare its Merkle root with every other node’s Merkle root. This allows for the building of light software clients that do not require storing the entire blockchain to validate their own transactions.

To calculate the Merkle root, you first create a Merkle tree, where the leaves are the transactions in the current block. Figure 2-5 shows the structure of a Merkle tree.

Figure 2-5. Flow chart of a sample Merkle tree

HA is the transaction (tx) hash of the first transaction, HB is the tx hash of the second transaction, and so on (we’ll talk more about cryptographic hashes in “Hashes”). HAB is the hash of HA + HB => HA+B = SHA256( SHA256 (HA + HB)).

By moving up the Merkle tree and generating hashes of all the leaves, you eventually reach the Merkle root (yes, the Merkle tree is an upside-down tree). If the number of transactions is odd, then the last transaction is replicated in order to continue this process. The Merkle root is an important value that helps to generate the block hash (see “Block Hashes”).

Figure 2-6 shows the Merkle root generated for a sample block, and Figure 2-7 shows the flow chart of this Merkle tree.

Figure 2-6. Overview of Bitcoin block #125552
Figure 2-7. Flow chart of the example Merkle tree

Here’s how we arrive at the Merkle root for this example:

This is the tx hash of the first transaction:

HA = 51d37bdd871c9e1f4d5541be67a6ab625e32028744d7d4609d0c37747b40cd2d

This is the tx hash of the second transaction:

HB = 60c25dda8d41f8d3d7d5c6249e2ea1b05a25bf7ae2ad6d904b512b31f997e1a1

This is the tx hash of the third transaction:

HC = 01f314cdd8566d3e5dbdd97de2d9fbfbfd6873e916a00d48758282cbb81a45b9

This is the tx hash of the fourth transaction:

HD = b519286a1040da6ad83c783eb2872659eaf57b1bec088e614776ffe7dc8f6d01

Thus:

HA+B = 0d0eb1b4c4b49fd27d100e9cce555d4110594661b1b8ac05a4b8879c84959bd4
HC+D = bfae954bdb9653ceba3721e85a122fba3a585c5762b5ca5abe117b30c36c995e
HA+B + HC+D = Merkle root = 
  2b12fcf1b09288fcaff797d71e950e71ae42b91e8bdb2304758dfcffc2b620e3

The important takeaway here is that the Merkle root can be used to quickly detect tampering in blockchain nodes. If there has been any tampering or corruption of transactions in the blockchain on any given node, its Merkle root hash will no longer match that of the other nodes.

Signing and Validating Transactions

Each transaction input contains a signature that provides proof that the owner of the sending address has authorized the transaction.  The signature is generated and encrypted using ECDSA, a cryptographic algorithm that takes the private key and transaction data as inputs, as illustrated in Figure 2-8.

Figure 2-8. Encryption process to generate a transaction signature

When all the nodes are verifying the transaction, they can easily verify the validity of the signature by using an ECDSA verify function, as illustrated in Figure 2-9.

Figure 2-9. Verifying the signature on a transaction

The important thing is that the private key is not required to check whether the digital signature authorizing the transaction is valid or not. Therefore, all nodes can easily validate the transaction using public information, but they can’t generate the signatures themselves because the private key is required for that.

The Coinbase Transaction

The first transaction recorded on every block is called a coinbase transaction. It is made up of two values:

Block reward
This is the reward a miner receives from the network for performing the work to discover a block and doing their part to provide processing power to the Bitcoin network. The reward comes in the form of new bitcoin being added to the world supply.
Transaction fees
This is the sum of all the transaction fees that are included in each transaction that gets added to the current block. There are often more transactions waiting to be processed than can fit into a block, generating a marketplace for transaction fees. The faster the miner wants a transaction to be processed, the higher the fee. The Bitcoin Fees site shows what current average transaction fees are.

The coinbase transaction has only one input, called the coinbase, which is blank. It also has some other special properties—for example, the previous transaction is 32 bytes of 0, and the script signature is permitted to contain arbitrary data that the miner can choose, such as the nonce header overflow (see “The mining process”).

Figure 2-10 shows an example of what a coinbase transaction looks like in a Bitcoin block explorer.

Figure 2-10. Sample coinbase transaction

Bitcoin Transaction Security

Bitcoin transactions are push transactions, meaning that the sender—the one pushing the funds out of an account they control—is the one to initiate the transaction. In contrast, a pull transaction is initiated by the receiver. An example is a credit card transaction: in this case, the merchant who is receiving the funds initiates the transaction.

Pull transactions are significantly less secure because they require the sender to share their account details with the receiver. To compensate for this weakness, pull payment networks (like Visa) provide chargebacks, or the ability to dispute a transaction and ask for a refund.

As push transactions, bitcoin transactions are significantly more secure. When initiating a transaction, a sender never has to reveal any of their account information. The only way a fraudulent transaction can take place is if an unauthorized person gets a copy of someone’s private key.

Note

The most common way for an unauthorized person to get hold of a private key is by breaking into an unsecured server or database.

With the technology available today, it is considered to be impossible to guess or reverse engineer what someone’s private key is. The only way to guess a private key is through brute force—trying every possible combination.

A private key is a 256-bit number, which means there are 2256 potential combinations to try:

  • 2256 = 1.1577 = 4 billion8

The combined total power of the Bitcoin network in 2020 is greater than that of any of the world’s supercomputers. Currently the bitcoin hash rate—an estimate of how many hashes are being generated by all the miners trying to solve any given block—is 90 exahashes per second. That works out to something like pow(2,128)/(90000000000000000000*3600*24*365)=119,892,034,120 per year. (The use of pow(2,128) is because ECDSA, the cryptographic algorithm used to generate a bitcoin private key, can be cracked in proportion to the square root of the key size.)

So, if you harnessed the processing power of all the miners in the Bitcoin network, it would take them this long to go through every combination:

  • 4,589,678,828,85137 years

Brute force attacks are commonly used to hack into computer systems, with the attacker trying a huge number of possible user passwords. Bitcoin private keys are resistant to brute force attacks because there are so many possible combinations to try. 

Hashes

A cryptographic hash is a function that encrypts any form of data into a fixed-size string. Hashes have the following attributes that make them attractive for blockchain:

There are many different cryptographic hash algorithms. Two of the most common are:

  • SHA-256, commonly used by Bitcoin

  • Keccak-256, commonly used by Ethereum

A common use case for a hash is a secure website storing a hash of your password in its database. Let’s say your password for the website www.store.com is FNj`{:;`k#F43rQ\.

For extra protection the website’s database will store not the password, but a hash of the password. If the website uses the hash function SHA-256, the resulting string stored in the database will be:

SHA-256("FNj`{:;`k#F43rQ\") =
6586BC035202DFF98A67B814ACA615E613CBBFAE8FFA8F4A475DA0FAEF079C9D

Then, when you log in, the website only needs to verify the entered password by comparing the hash of the string you typed in to the hash stored in its database. This process makes the website more secure because if a hacker breaks into the database, they’ll only get the hashes of customer passwords.

Block Hashes

A block hash is a snapshot of what the entire blockchain looked like at the moment that block was created. In accounting terms, it’s like a balance sheet for the entire network. Every node in the network refers to the block hash to verify that its view of the network is the exact same as everyone else’s (see Figure 2-11). If there’s even one minor difference in a node’s ledger, its hash will look significantly different. This is what makes blockchain tamper-evident; if the content experiences tampering or corruption, the resulting hash will no longer be the same.

In Figure 2-11, for example, Anonymous #4 has a different block hash than every other node—which means that node’s view of the network is wrong. Verifying the block hash is a much faster process than each node checking what every other node’s history of transactions would be.

A Bitcoin block hash is generated using a double SHA-256 hash function on the Block_Header:

SHA256( SHA256( Block_Header ) )

A Block_Header is made up of the data shown in Table 2-2.

Table 2-2. Anatomy of a Bitcoin block
Field Description Size (bytes)
Version Block version number 4
hashPrevBlock 256-bit hash of the previous block header 32
hashMerkleRoot 256-bit hash based on all of the transactions in the current block 32
Time Current block timestamp as seconds since 1970-01-01T00:00 UTC 4
Bits Current target in compact format 4
Nonce 32-bit number (starts at 0) 4

The two most important fields in the Block_Header are hashPrevBlock, which provides a snapshot of what the Bitcoin network looked like in the previous block, and hashMerkleRoot, a snapshot of all the transactions included in the current block.

Figure 2-11. All nodes in the network maintain the same view of the state of the network by having the same blockhash

Including the previous block hash when generating the new block hash ensures that every block is connected, or “chained,” to the previous block, as shown in Figure 2-12—hence the name blockchain.

Figure 2-12. Block hashes connect successive blocks together in one big chain

Custody: Who Holds the Keys

In financial services, custody refers to the ability to hold, move, and protect assets. It’s a good idea to know the concepts that support custody because of the number of different ways there are to store crypto assets.

Wallet Types: Custodial Versus Noncustodial

Similar to how people usually store value such as cash and credit cards in a folding piece of leather, cryptocurrency is stored in what is also known as a wallet. In this case, however, it’s really just an interface for storing cryptographic keys and keeping them secure; the cryptocurrency itself does not physically exist on any device, and the wallet is used exclusively for storing the keys associated with it. Many people say crypto and blockchain are “secured by math,” and this is what they are talking about.

In general, there are two types of cryptocurrency wallets: custodial and noncustodial. A custodial wallet is controlled by a trusted entity, with the user typically having to access its contents via a web interface. These sites store private keys for users; this way, users don’t have to worry about them.

Exchanges are a common example of custodial wallets—they hold your cryptocurrency in an account, and they own and control the keys. One popular example is Coinbase, which was founded in 2012 and is one of the oldest and largest custodial wallet providers in the market.

The obvious downside of this arrangement is that if the exchange goes bankrupt or runs away with the funds, there’s nothing the user can do because they don’t directly own or control the keys. Trust is therefore an important issue: users must trust the service to safeguard their keys and act responsibly.

Noncustodial wallets give users complete control of keys. However, there is a downside here too. The user is entirely responsible for the security of their private keys. If they lose them, that could result in complete and total inaccessibility of their funds. Blockchain.com, founded in 2011, is one of the oldest and largest noncustodial wallet providers in the market.

Note

Using noncustodial wallets takes effort and is not recommended for people with no experience in computer security, but it can be done with fairly simple steps. We’ll talk more about securing noncustodial wallets in “Security Fundamentals”.

Wallet Type Variations

The two primary wallet types can be implemented in a variety of ways:

Security Fundamentals

It should go without saying that it’s important to keep private keys private. If someone is able to access your private key, it won’t be difficult for them to sign transactions on your behalf and empty out your wallet.

One of the most common ways people lose crypto funds is through authentication issues. That means exposure of either private keys or passwords associated with a cryptocurrency wallet. There are a few ways to help prevent this from happening:

Identity verification
With so many different messaging services being used today, it can be hard to tell whether someone is who they say they are. Make sure to verify through personally known information who you are talking to, especially if someone starts making strange requests—like asking for cryptocurrency.
Two-factor authentication
In addition to using a password, turning on two-factor authentication is a good idea. Two-factor authentication requires another source for verification, such as when a website sends a text message to your phone containing a code you must enter in order to access your account on the site. There are multiple ways of doing two-factor authentication, and some are more secure than others. SMS verification using an app like Authy or Google Authenticator is one way. It can also be done via a hardware device like the YubiKey that plugs into the user’s computer. Note that the first option can be susceptible to porting (see below) if not done properly.

Types of attacks to watch out for include:

Cell phone porting
Porting is a common type of attack where someone takes over your cell phone’s number, allowing them to intercept incoming messages. This is often accomplished by calling the carrier and providing some personal information the attacker has learned. Because of the dangers of this attack vector, it’s best not to use SMS verification for two-factor authentication. A good alternative is to set up a portable VoIP phone number that supports text, like Google Voice.
Phishing
Phishing is a very effective way for hackers to take control of accounts (and cryptocurrency). The attacker typically claims to be from a familiar and trusted organization, like a government agency or a well-known company, and sends the user a message containing a link that encourages them to reveal personal information, such as a password. This might be a spoofed email that looks like it’s from your boss and asks for your Social Security number, or the attacker could even ask an innocent question first to establish trust. Although it may sound far-fetched, phishing is the most effective tool attackers have—all they need is to obtain some personal information. The best policy is to never give personal information out to anyone in response to an unverified email or text. Always try another means of communication, like a phone call, to verify the request as valid.

It’s important to keep these fundamentals in mind. Even Steve Wozniak, the cofounder of Apple, was scammed out of bitcoins by letting a prospective buyer use a credit card to send funds to him via PayPal. The card transaction was reversed, but the bitcoin had already been sent. Be very careful, as cryptocurrency is a major attraction for scammers!

Recovery Seed

A recovery seed is a series of words that can be used to retrieve a private key stored in a noncustodial wallet. Seeds are commonly used as a memory aid because it is very difficult to remember a private key, which is just a string of random numbers and letters. Seed phrases usually store enough information to allow the user to recover their wallet. An example seed phrase might look like this:

witch collapse practice feed shame open despair creek road again ice least
Note

It’s very important to store the seed phrase for a noncustodial wallet somewhere safe. Indeed, for practical purposes, the recovery seed phrase is your “wallet,” and with it an attacker can easily access your funds. Cryptocurrency gives you complete control over your balances, but it’s up to you to secure them.

If you record your seed on paper, be sure to laminate it or otherwise make sure the writing does not fade. Using an etched metal seed storage device like the one shown in Figure 2-13 can also be useful, but it’s important to consider factors like corrosion or humidity.

Figure 2-13. Sample cold storage, embedding recovery seed onto metal (image credit: http://www.coldti.com)
Tip

The most common mechanism for generating a mnemonic to use as a wallet seed is BIP39, the standard for creating phrases from addresses.

Cryptocurrency can and has been lost, whether a user controls their private keys or not. It’s important to use secure communication tools, set up two-factor authentication, have a PIN with a cellular carrier, and be aware of phishing. Once cryptocurrency leaves a wallet, it’s almost impossible to get it back.

Mining

In the beginning, cryptocurrency mining was solely a hobbyist’s pursuit. Early adopters who wanted to support the Bitcoin network downloaded and ran the Bitcoin Core software, and they were able to mine a few bitcoins here and there just by running the software on their computers.

In 2010, 1 bitcoin was worth $0.30, so miners weren’t making much money—it was just a hobby. Block rewards at that time were 50 BTC, so if a miner discovered 1 block a day, they’d earn $15. However, that changed over time as the hobbyists gave way to serious professionals.

Mining Is About Incentives

Over time, as the price per bitcoin grew and interest in more professional mining hardware resulted in new equipment, the “difficulty” of mining also went up. It did not take long before just using a regular computer to mine was not enough. Miners needed special computer hardware known as graphics processing units (GPUs) to compete. Then they started using special microprocessors called application-specific integrated circuits (ASICs) to improve efficiency. Today, most cryptocurrency mining is done in huge data centers, with racks upon racks of machines requiring large amounts of power and cooling. So how did we get here?

It’s all about incentives. In the beginning, miners were competing with one another using personal computers to solve what can basically be called puzzles. The reward for doing this was 50 brand-new bitcoins—and a new block would be published to the chain. Over time, however, the crypto rewards for solving these “puzzles” turned into serious revenue (Figure 2-14).

Figure 2-14. Mining revenue in bitcoin is as volatile as its price

There are massive benefits to mining bitcoins at scale. With access to cheap power and data centers, mining cryptocurrency can be profitable. As a result, mining has mostly moved beyond the purview of the hobbyist. New cryptocurrencies may arrive that welcome hobbyist miners, but Bitcoin has reached an enterprise level of large-scale data center–based mining.

Consensus

Consensus is a way of reaching agreement between various participants who have shared values and goals, and it is an important component of how blockchain networks succeed. Though there are other ways of achieving consensus, in blockchain the two most popular are proof-of-work and proof-of-stake. Since we are focused on cryptocurrency and mining, this explanation will concentrate on proof-of-work and how it applies to Bitcoin. Note that enterprise applications that use blockchain generally do not use proof-of-work consensus and do not require miners.

Proof-of-Work

Proof-of-work enables cryptocurrency transactions to be confirmed and blocks to be published on the Bitcoin blockchain. First described in a research paper by Markus Jakobsson of Bell Labs and Ari Juels of RSA Laboratories, proof-of-work was initially created to bind economic value via computer processing to otherwise free services, like email, in order to stop spam. Because proof-of-work requires computing power, it reduces the incentive to attack or flood a system. The economic value provided in proof-of-work is directly correlated to the price of the electric power that is used in the mining process.

In proof-of-work cryptocurrency mining, a hash function is used to verify data. A hash is output on the blockchain as public proof using a hash algorithm. The computer speed at which this is done is known as the hash rate. With many cryptocurrencies, proof-of-work–based computer power is what secures the network—and that power has become quite substantial. Although hash rates fluctuate, Bitcoin has surpassed 70 million terahashes per second in the past (see Figure 2-16).

In cryptography, many different types of proof-of-work have been devised. For cryptocurrencies, a few are used. Bitcoin uses the SHA-256 hash algorithm, for example, while Litecoin uses a more memory-intensive cryptographic Scrypt algorithm.

Figure 2-16. History of the hash rate for the Bitcoin network

Block discovery

About every 10 minutes, a new block of bitcoin transactions is confirmed by one miner. Since there are thousands of miners in the network, the network needs to achieve consensus on which miner gets the right to confirm the new block.

All a miner has to do to discover a new block is generate a Bitcoin block hash that is considered valid by the network, using the following criteria:

  1. It is a hash of a valid block header.

  2. The resulting block hash is a number that is lower than the current network target.

The target is a constantly changing number that must always be higher than a valid block hash. The difficulty is the average number of attempts required to discover a valid block hash. The network hash rate refers to how many times per second the miners in the network collectively attempt to generate a valid block hash.

The goal for the network, set in Bitcoin’s initial parameters, is that a new valid block should be discovered approximately every 10 minutes. Over time, the number of miners using computer processing power to discover a block changes along with variables like electricity use and processing power, among other factors. The processing power they are consuming is called the hash power. The miners’ computers are consuming this power to try to generate a valid Bitcoin block hash.

When the hash power of the Bitcoin network increases—that is, when more computer processing power is being applied to generate a valid block hash—it naturally takes less time for the network to discover a block. Therefore, in order to maintain an average of 1 block being discovered every 10 minutes, the Bitcoin network changes the network target to make it more or less difficult for the network of miners to discover a valid block hash.

The target value when the first Bitcoin block was generated was:

00000000ffff0000000000000000000000000000000000000000000000000000

This is the highest possible target value. When you compare the first block’s hash to this, you can see that it is a (hex) number lower than the target at that moment:

Initial target 00000000ffff0000000000000000000000000000000000000000000000000000
Block #0 hash 000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f

A simple way to see which number is lower in hex is by counting how many zeros are at the beginning of the hash. The initial target has 8 zeros, whereas the first block hash has 10 zeros; therefore the block #0 hash is a lower number, and therefore it is valid.

When block #0 was discovered, there was little competition in the Bitcoin network to discover a block. So, the target value was high. The difficulty at that moment was 1, meaning that on average it would require 1 attempt to generate a valid hash. Ten years later, there are thousands of miners consuming significantly more hash power to discover a block. Therefore, the target 10 years later is a lower value, requiring more attempts.

Compare the target with a valid block hash from July 28, 2019:

Target 0000000000000000001f3a08000000015667e3e2c52a81e977a0b71f70e5af97
Block #587409 hash 0000000000000000000001f57b098911a90b164b9812304f4f7615cf9f91f66a
Difficulty 9,013,786,945,891.68 estimated attempts required to discover a valid block hash

Bitcoin is designed to have a new target recalculated by all the nodes in the network every 2,016 blocks (approximately 14 days). The new target value is calculated as the target value that would have generated the previous 2,016 blocks at intervals of exactly 10 minutes. This is the Bitcoin network’s way of self-correcting the difficulty required to generate a valid block hash by the miners that participated in the network over the previous 2,016 blocks.

The main reason this difficulty process is in place is to ensure the supply of bitcoin is predictable and follows a specific schedule (see Figure 2-17). When each new block is created, new bitcoin is created as well, although over time this supply is diminished. With bitcoin, the size of the reward is also designed to get smaller as supply fills. Every 210,000 blocks, or approximately every 4 years, the block reward is cut in half. It went from 50 bitcoins to 25, then from 25 to 12.5, and so on, and will continue decreasing until 2140, when roughly 21 million total bitcoins (a hard cap) will have been mined.

Figure 2-17. Bitcoin supply over time

The mining process

At every moment, hundreds of thousands of miners on the Bitcoin network are competing to discover the next valid block on the blockchain. Miners are incentivized to do this because of the block reward and transaction fees. As mentioned earlier, the miner needs to make sure the following are true to generate a valid block hash:

  1. It is a hash of a valid block header.

  2. The resulting block hash is a number that is lower than the current network target.

To generate a valid block hash, the miner needs to input the information shown in Table 2-3.

Table 2-3. Contents of a valid Bitcoin block
Field Description
Version The Bitcoin client version that the miner is currently using
hashPrevBlock The hash of the last block that the miner sees at this moment
hashMerkleRoot The hash of all the transactions the miner decides to include in the current block
Time The block timestamp, calculated as seconds since 1970-01-01T00:00 UTC
Bits The current Bitcoin network target
Nonce Starts at 0; if the resulting hash is not valid, then add 1 and try the new hash

All the fields except for the nonce are taken from public sources of information. When a miner begins trying to discover a valid block, they initially set the nonce to 0 and then try to generate a hash that matches the block hash, which is randomly generated. Miners try over and over and to find this block hash, using hash power, meaning the more efficient a miner is at generating these hashes, the higher a hash rate they have.

If the resulting block hash is invalid, the miner adds 1 to the 32-bit nonce and generates a new block hash, which it hopes will be valid. If the miner runs out of nonce space, known as an overflow, they use script sig space in the coinbase transaction. If a miner finds a block hash that is valid, which includes satisfying the target criteria, then they have discovered a valid block. The process of continuously trying new block hashes is the proof-of-work every miner puts effort into, as shown in Figure 2-18.

Figure 2-18. The proof-of-work process miners go through to attempt to discover a new block

After a miner discovers a valid block hash, the miner then broadcasts that new block hash to all the other miners in the network. There is a possibility that two different miners will discover a valid block and broadcast the new blocks to the network at the same time. It is then up to all the other miners in the network to achieve consensus on which new block will be added to the blockchain, as Figure 2-19 illustrates.

Figure 2-19. An event when two miners discover a new block at the same time

Before a miner adds a new block to the blockchain, the miner verifies that the following are true:

  1. The block is valid.

  2. All the transactions in the block are valid. This includes confirming the data signatures used to unlock transaction outputs.

Consensus is achieved when more than 50% of the other miners in the network include the same new block into their copies of the blockchain. The miners collectively “vote” on which block they recognize to be added at that moment to the blockchain, as well as verifying that all the transactions are valid.

Transaction life cycle

In a centralized payments system like PayPal, the life cycle of a transaction is pretty straightforward. You log in to the PayPal website or app, enter the transaction details, and press Enter. If PayPal responds with a “Completed” notification, then your transaction has been processed and you are done.

The transaction life cycle in Bitcoin is very different, and there are many different steps involved in the process. The following are the four main stages a transaction goes through:

  1. Broadcast. The first step is generating a valid bitcoin transaction and then broadcasting the transaction details to the Bitcoin network. Most Bitcoin users will use an online wallet (such as Coinbase), and in the background there is software running a client connected to the Bitcoin network that enables them to see these transactions.

  2. Unconfirmed/Mempool. As every miner in the network receives the transaction, it places that transaction into its memory pool, or mempool. The mempool is a collection of all the bitcoin transactions that are in an unconfirmed state and are still considered active. By default, if a transaction has been sitting in the mempool for more than two weeks, it is considered inactive and is dropped from the mempool.

  3. Confirmed by miner. When a miner discovers a new block, the miner decides which transactions to include in that block, choosing from transactions that are sitting in the mempool. Miners choose transactions in order of transaction fees, starting with the highest ones. A transaction is considered confirmed by a miner when that miner adds a block containing that transaction to its blockchain. However, that miner cannot see the other miners’ copies of the blockchain, and there is no assurance that the same block of transactions has been added to their chains.

  4. Confirmed by the network. As a block is buried under newer blocks, the chances that the Bitcoin network has achieved consensus to include that block increase. A transaction is considered to be confirmed by the entire Bitcoin network when the network has achieved consensus to include the transaction’s block in the blockchain.

Confirmations

Bitcoin wallets, and most people in the industry, consider a transaction to be safely confirmed by the network when that transaction has reached at least six confirmations. A confirmation involves a miner adding a block that contains transactions to the chain. Figure 2-20 illustrates the decision process of miners for including a block.

Figure 2-20. Example of proof-of-work miners deciding on which block to include in the blockchain

In Figure 2-20, block #100 has reached three confirmations. All four miners in the network have included the same block. At block #101, three of the miners (75%) have included the block discovered by Bob, but one (miner D) has included the block discovered by Charlie. At this moment, miner D does not realize yet that their view of the blockchain from block #101 on will have to change. If there is a transaction in miner D’s block #101 that is not in the other miners’ block #101, that transaction will not be included in the network’s blockchain. This is why the more confirmations a transaction has, the more likely it is to be included in the Bitcoin network’s blockchain.

Note

Many services have different cryptocurrency confirmation schedules. For example, some services require as few as three Bitcoin network confirmations before a transaction is considered complete, although the standard is usually six confirmations. Some services may require even more confirmations, depending on a variety of factors (including the type of cryptocurrency used).

Proof-of-Stake

Proof-of-stake is a consensus algorithm that aims to improve on proof-of-work by eliminating the need for mining. Instead, holders of a cryptocurrency “stake” their balances to gain voting rights and have a chance of being selected by the network to validate transactions. Staking therefore allows you to act as a node, or validator. Though there are no expensive hardware requirements or difficult computational processes to worry about, there are economics in play with the cryptocurrency holders putting up funds. And there are incentives: those who stake are provided rewards in proportion to their holdings.

Instead of selecting the miner who can discover a valid block hash first by doing work, proof-of-stake consensus selects a node based on various staking criteria, which may include size of stake (the amount staked), age (which address has held onto the cryptocurrency for the longest time), wealth (which address has held onto the most cryptocurrency for more than X amount of time), etc. Alternatively, in some systems an address that has staked some of the cryptocurrency is selected at random.

Pros

  • Because there is no mining, little energy is consumed in creating blocks, so less energy is wasted powering the network.

  • Gives more control to those who are more invested in the network.

Cons

  • Control of the network is tied to the distribution of wealth in the network. Most cryptocurrency wealth is concentrated among a small group, and therefore control of the network is more centralized than in a proof-of-work network.

  • By giving more control to those who are more “invested” in the network, rather than the ones who are doing more “work,” proof-of-stake could lead to more divergence between the rich and the poor than proof-of-work.

There has been a lot of criticism of the proof-of-work model. Although a novel idea for cryptocurrency when Satoshi Nakamoto proposed it, the hardware arms race to develop the most powerful ASIC has arguably made proof-of-work more resource-intensive than it should be. One side of this argument says Bitcoin has become an environmental problem because the amount of electricity required to confirm transactions and generate new bitcoin is no longer economically efficient. Another side points out that most of the electricity consumed by Bitcoin is actually from renewable sources like hydroelectricity, which is where miners get cheap power.

Determining who gets to generate the next block in proof-of-stake systems is very different from in proof-of-work systems, and there are a few different ways to do it. Some cryptocurrencies use randomized block selection, which is based on a combination of stake size (the higher the better) and hash value (the lower the better). Another method is coin age–based selection, which is based on the number of coins staked and the number of days the coins have been held (possibly combined with the randomized selection method). The latest method of securely introducing randomness is by using the RANDAO random number generator and verifiable delay functions (VDFs) together.

A few cryptocurrencies currently use staking, including Dash, Neo, and Tezos, but it’s still a relatively new concept and has not yet seen widespread use like proof-of-work. There are criticisms that proof-of-stake is not a secure consensus mechanism, as a fork would in theory create equal incentives on two different chains (forks, which occur when a blockchain diverges, are covered further in Chapter 3). There is also the risk of a “fake stake” attack, where a staker with little to no balance could disrupt a network, as validation is much more complex to complete than with proof-of-work.

Another theoretical safety issue is the Nothing-at-Stake problem, where a miner freely creates several blocks, causing forks and denial-of-service attack possibilities. This has typically been addressed with slashing algorithms to reduce stake in order to penalize badly behaved validators.

Other Concepts for Consensus

There are other ideas outside of proof-of-work and proof-of-stake. Achieving consensus is still a new and evolving technology concept, and different ideas are being tested. This is why consensus algorithms are frequently decoupled from blockchains—in other words, the blockchain technology itself is not tied to one particular method of consensus. This way, third parties can build and market consensus algorithms to be added to commonly used blockchains.

Cryptocurrencies like Ripple and Stellar are active projects that use some very different types of consensus protocols, although it can be argued these systems are not entirely distributed. Both use what is known as Byzantine agreement, a way for distributed nodes to cooperate to confirm transactions. Many of these nodes are controlled by the projects themselves, so they may seem centralized. However, both Ripple and Stellar bypass traditional banking and payments systems by having a blockchain-based unit of account. This allows users to save on the costs usually incurred in traditional financial systems.

Alternative methods

One approach being looked at is called proof-of-storage. Something that can be done computationally and is also resistant to ASICs, proof-of-storage uses the validation of storage through filesystems to verify transactions. Projects such as Permacoin, Torcoin, and Chia are utilizing a version of proof-of-storage.

Another interesting approach to scaling is proof-of-history. For example, in proof-of-work, Bitcoin can be reimagined as a clock that ticks every 10 minutes (thanks to the difficulty adjustment), and the ordering of messages is not agreed upon until 10 minutes pass and the network reaches consensus. In proof-of-stake, everyone is receiving messages from the network, and they need to communicate with other’s nodes to agree on the ordering and timing of these messages before they finally reach consensus, and the network progresses.

Proof-of-history is a way to create objective timestamps that nodes within the network can rely on, enabling them to optimistically trust the ordering and timing of the messages before consensus is reached. Consensus then comes in later: participants within the network vote on what they believe to be the main branch, and each time they do this, they commit to not voting on another branch for a set amount of time. The more they vote for a particular branch, their commitment to not vote on another branch grows exponentially. They’re incentivized to consistently vote for what they believe to be the major branch because until they have stacked 32 votes for one particular branch, they won’t earn any network rewards.

Note

In her paper “Practical Uses of Synchronized Clocks in Distributed Systems”, Barbara Liskov of the MIT computer lab states, “Synchronized clocks are interesting because they can be used to improve the performance of distributed algorithms. They make it possible to replace communication with local computation.”

Delegated proof-of-stake consensus is an energy-efficient form of consensus where users delegate, placing tokens with a “candidate,” and use voting to help govern the network. There is also voting-based consensus, and lottery-based consensus is used in more private implementations of blockchain.

Stakeholders

In addition to a protocol network, wallets, and miners, there are other stakeholders in the cryptocurrency ecosystem. These may be centralized services or for-profit businesses, and they provide important functionality needed in the ecosystem. There are five categories of stakeholders that a typical user might interact with in cryptocurrency: brokerages, exchanges, custody services, analytics services, and information providers.

Information

The blockchain industry is changing on a daily basis. New companies, new ideas, and brand-new cryptocurrencies seem to pop up all the time. CoinDesk, which was founded in 2013, is one of the oldest and largest organizations dedicated to providing news and other research on the industry. Major publishers like the New York Times, the Wall Street Journal, and Bloomberg also do a good job of covering the cryptocurrency industry with dedicated beat journalists.

Conferences are great educational resources, but can be expensive. For the budget-conscious, Meetup.com is a great source to find local cryptocurrency events. Using the search terms bitcoin, ethereum, or blockchain will usually turn up some local meetups, most of them featuring speakers talking about current events, best practices, or interesting technical topics.