Overlays will provide a service layer for information in the Teranode era

Getting your Trinity Audio player ready...

How do you verify valid transactions on a blockchain, if that blockchain processes a million of them every second? Can a business check only transactions relevant to its operations, while ignoring all others? The answer is yes, by using Overlays and services to identify transaction outputs by type. Read on to find out more about what Overlay Networks are, and how they work.

The advent of Teranode and its massive data processing capabilities mean that current methods used to identify and separate relevant transactions from the set of all transactions may need to change. “Global listening” involves taking a stream of transactions as they are announced, and using a filter to sift through and identifying which of those are relevant to a certain business or project. Examples of this method are GorillaPool’s JungleBus and the older BitBus by Planaria.

This method makes sense if you already have to process all transaction, if you are a miner like GorillaPool. What if you are a service business who is not themselves a miner?

The weakness of global listening in that case is, as the network scales (with total transactions and amount of blockchain data growing), more bandwidth is used and wasted. Additionally, the filtering method must also scale. Teranode will scale the BSV network exponentially, and global listeners won’t be able to keep up. It will become financially prohibitive for any project to ingest all transactions the network processes without a corresponding fee.

So the main aim of Overlays is to allow businesses to maintain their own sets of unspent transaction outputs (UTXOs), i.e., only those relevant to their operations. Many projects try to do this already in some form, using techniques they’ve developed themselves for their own purpose, and we now in hindsight refer to these systems in general as Overlays.

Mining nodes must keep the full UTXO set to ensure they can validate any transaction. Overlays need only maintain their own subset.

That’s where Overlays come in

An Overlay instead uses SPV (simple payment verification) and Merkle Paths rather than full block data. To give an idea of how this scales more efficiently, imagine a business that processes one transaction every 10 minutes (573 bytes). Observing and filtering the full transaction data would involve 340,000,000,000 bytes of information. With SPV, that’s reduced to just 1,537 bytes—making SPV 221 million times more efficient at Teranode scale.

SPV isn’t a new concept. In fact it’s detailed in the original 2008 Bitcoin white paper as the only way for a blockchain like this to operate at scale. SPV uses Merkle proofs to determine whether or not a transaction has been included in a block, using only data from the Block Header.

Overlays are a way to create a shared context by maintaining a subset of transactions, and allowing users to read data from chain using SPV. They also provide a framework for the monetization of distributed systems presenting new economic opportunities for service providers. For example, a service could track UTXOs associated with tokens, or a specific kind of token (loyalty points, event tickets, or even a CBDC). Wallets like HandCash or RockWallet could use overlays to track only UTXOs associated with their services. These specialized services would ideally combine their UTXO sets with search and retrieval functions, charging microtransaction payments for each service they perform for clients. Projects built on the BSV blockchain could leverage and utilize these services instead of having to develop and maintain their own overlays.

How to identify transaction types

How do you categorize UTXOs? There are several ways to identify specific UTXO types, based on their structure. This could be data contained in a simple OP_RETURN, or simply through tagging metadata.

Project Babbage’s Confederacy Lookup Services is an example of how Overlays would function. It categorizes UTXOs with “Topics,” feeding them into its custom data stores. It also allows the distribution of read load across multiple hosts on the same topic, and solves open protocol service discovery across a p2p overlay network.

Jake Jones, Head of Network Infrastructure at BSV Association, recently spoke about Teranode and the need for Overlay Networks at the sCrypt Hackathon event. His full presentation on the topic is in this video:

Overlays are another example of how the BSV protocol rules allow for greater specialization within the blockchain industry. Miners/Nodes keep the network secure by processing all transactions; Overlays provide a service layer for applications to share context and retrieve relevant subsets of the UTXO set. This specialization is necessary to keep a working, blockchain/Web3-based digital economy functioning, ever more so as blockchain scales to meet the entire world’s data needs.

Watch: Teranode is the future of the BSV network