unwriter’s tools explained, use cases reviewed: Part 2

Getting your Trinity Audio player ready...

Pseudonymous developer unwriter has continued to build tools that help streamline application development for those looking to build on top of Bitcoin SV (BSV). He has released many tools since May of 2019, and since my presentation at CoinGeek Toronto.

https://www.youtube.com/watch?v=81txTx6bhyI

This article will review each tool released since then at a high-level and review a potential use case for each one (in a format similar to a previous article I wrote) in hopes of summarizing what unwriter has been up to in the latter half of 2019.

The first set of tools reviewed here are more low-level, plumbing type tools so the use cases are more developer focused as opposed to business, end-user focused.

This is the first installment of two parts, another article in a similar format will be released the following week.

Before diving into each tool, take note of the above image from unwriter’s Overpool documentation. This image represents the vision they have for how their tools interact with Bitcoin. Please consider this image as you read through this article.

Neon Planaria

I covered the old Planaria in my presentation but it has evolved drastically since then. unwriter has now broken it into two parts—Neon Planaria and Grid Planaria.

Neon Planaria is the same as the original Planaria in terms of functionality. The primary difference is that it is now more lean—it has less code written and eliminates the setup previously required. It implements three event handler functions, just as the old one did:

– OnStart – called when the application starts
– OnMempool – called when a new transaction arrives in the mempool listened to
– OnBlock – called when a new block is mined

Neon Planaria also provides an API for yourself or others to interact with (named Planarium). Developers can leverage the functions here:

– OnStart – called when the application starts
– OnQuery – called when a query is made to the API

Additionally developers can implement their own custom JSON query language as opposed to only bitquery. This is powerful because the Neon Planaria instance can implement some JSON query that other existing applications already use—or they can implement a particularly robust one that other apps can use in the future, which can lead to a streamlined path for applications interacting with each other.

Use case

Create any custom, filtered database in any format from Bitcoin transactions. This framework streamlines the development required for this, and we no longer have a requirement to use MongoDB only. Additionally, developers can aggregate Bitcoin transactions and easily serve them to others over HTTP.

Grid Planaria

The key difference between Grid and Neon Planaria is that Grid Planaria interacts directly with a Bitcoin node, focused on transforming raw Bitcoin transactions into some efficient data format. Not only is it responsible for delivering the data efficiently, but it stores the data in a readable format as well.

Applications should use Neon Planaria, whereas more heavy duty data processing and formatting applications should use Grid Planaria. Grid Planaria implements BOB serialization, which will be reviewed later and is more robust and scalable than the previous TXO format.

Use case

Miners can run Grid Planaria as augmented software alongside their nodes as a means to monetize the data for others. This just makes sense since they are already needing to run a node for their business, providing an incentive for them to keep the data instead of pruning.

Implementing this along with Neon Planaria establishes a framework where miners can begin to serve this data to consumers. Leveraging the built-in APIs, they can understand which data is being requested most often in order to determine which data is valuable enough to keep and can prune the rest. These tools when used by miners truly manifest the economics of Bitcoin.

Bitwork

Bitwork is a library that abstracts processing data from the Bitcoin network. It removes the complexities of dealing with the low-level P2P protocols and JSON-RPC to streamline methods for developers to read data directly from a node. The library has functions to parse and map data as well. As unwriter writes:

“Bitwork lets you treat Bitcoin like HTTP.”

Developers can specify which node to connect to and fetch data from, which can lead another path to monetization for miners. Two modes are provided:

– Fetch – get data from Bitcoin (one-time query)
– Listen – event handler to do some action once a transaction arrives at the node

BPU – Bitcoin Processing Unit

BPU is a means for developers to define their own transaction format as opposed to just using the TXO format which was previously used in unwriter’s tools. The BPU library can deserialize Bitcoin transactions into some readable format. Inputs, output and OP_RETURN push data can be structured into a consistent format. This library provides the functionality for developers to customize how they want the Bitcoin transactions to be formatted for their application.

Use case

Build custom transaction format that an existing application can easily digest. If the application being integrated with supports some format such as CSV or JSON, this library can be leveraged to convert transaction data to that.

The ERP system I work with (Microsoft Dynamics 365) has standard functionality where external applications can post some JSON or CSV file to an HTTP endpoint. The system can then take that data and process it, creating a sales order for example. In this scenario the BPU library would help to morph Bitcoin transactions in a meaningful way that reduces development effort on both the ERP and integrating application sides.

BOB – Bitcoin OP_RETURN Bytecode

BOB is a new serialized format for OP_RETURN Bitcoin transactions. As mentioned, all _unwriter tools previously used the straight-forward TXO format.

BOB formats transactions in a more flexible, optimal and scalable way. TXO’s flaw was providing fixed locations for inputs, outputs, block attributes and OP_RETURN push data. Once applications began to concatenate Bitcom protocols, querying TXO became unsustainable.

The below example illustrates how we would query for multiple B:\\ files with TXO. We would need to know the fixed positions (s2, s6, s11) before querying which is in general, horrible practice (akin to hard-coding).

Instead of fixed positions, BOB lets us query for each instance of a B:\\ file. A simple query for the B:\\ protocol prefix will return all instances where it was used.

Before we would have found only pure B:\\ transactions, now with BOB each instance of data will be returned even if it is used in a subset of other transactions, or in different push data positions. BOB has the added benefit of solving the OP_FALSE OP_RETURN upgrade for Genesis since it no longer relies on the OP_RETURN opcode being in a certain position.

Use case

Any developer using unwriter’s tools should leverage BOB. BOB is a more scalable format and will be much better practice to implement in applications. Nesting of protocols are now more supported. Before with TXO if you needed to establish links between B:\\ files, a naïve implementation may have broadcast two B:\\ transactions with an added field on the second that references the first.

Now you can nest them in one transaction, and a simple query like above would return both files.

Summary

Even with pausing for fundraising, unwriter released many useful tools over the second half of 2019 that many application developers in the BSV space are using. unwriter has developed a vision for how they see the application ecosystem around BSV unfolding, and these tools will be crucial in paving the way towards that.

In my opinion, I believe these tools will end up being adopted by miners themselves in order to find other means of monetizing their infrastructure besides just hashing and finding block rewards, thus alleviating the persistent concerns around the incentives of data pruning.

The only way Bitcoin succeeds is by miners stepping up. And the software has been finally implemented to let them do exactly that.

Will the real Bitcoin miner please stand up?

— _unwriter (@_unwriter) July 23, 2019