The journey to scaling Bitcoin

This article was first published on Dr. Craig Wright’s blog, and we republished with permission from the author.

In 2012, I began a project to create what we originally called “iDaemon,” or the “Intelligent Daemon System,” which presents an elaborate design and architectural blueprint for a complex software system designed to scale Bitcoin to over 1 million transactions a second. This project encompasses various subsystems managing different aspects of transaction processing, accounting, banking, exchanges, messaging, and contracts in the context of a blockchain network, particularly Bitcoin. It was one of over 100 projects my companies were running.

The system is now being tested. We are demonstrating 1.5 million transactions per second. When I started the project in 2012, I vastly underestimated what would be needed. US$900 million later, we have a system that works. When I watched the project, the budget was US$20 million for two years. Sometimes, good things take a more protracted approach to be achieved.

1. Introduction to Intelligent Daemon System

The Intelligent Daemon System is an intricate framework specifically engineered for Bitcoin, designed to enhance its functionality as digital cash. This system integrates a range of subsystems, each meticulously crafted to address specific aspects of Bitcoin transaction processing and management. The architecture of the system is founded on principles of efficiency, security, and scalability, ensuring a robust and reliable environment for Bitcoin transactions.

One of the critical components of the system is its transaction management mechanism. This mechanism is optimised for handling Bitcoin transactions with precision and speed, catering to the unique requirements of Bitcoin as a digital/electronic cash platform. It incorporates state-of-the-art algorithms and protocols to ensure the integrity and security of transactions, a critical aspect given the distributed nature of Bitcoin.

Another significant element of the Intelligent Daemon System is its contracts management subsystem. This subsystem is designed to leverage the blockchain technology underlying Bitcoin, facilitating the creation and execution of smart contracts. These contracts extend the functionality of Bitcoin beyond mere transactional capabilities, enabling a variety of financial applications and services to be built on the Bitcoin platform. The inclusion of such a feature underscores the system’s commitment to not only managing Bitcoin transactions but expanding its utility in the digital economy.

Overall, the Intelligent Daemon System represents a significant advancement in the domain of Bitcoin transaction processing. Its comprehensive approach and innovative features make it a pivotal development in enhancing the functionality and usability of Bitcoin as digital cash.

2. Architecture Overview

The architecture of the Intelligent Daemon System, detailed in the document, presents a sophisticated blend of Service-Oriented Architecture (SOA) and Microservices, each serving a distinct purpose in enhancing the system’s functionality and adaptability. The SOA framework provides a robust foundation, facilitating the interoperability and independence of various services within the system. This framework is crucial for ensuring seamless integration and communication among different components, thus enhancing the overall efficiency and coherence of the system.

Delving deeper, the Microservices architecture plays a vital role in refining the system’s capability to manage intricate processes. Each microservice in the system is designed to execute a specific function, operating autonomously yet harmoniously within the overarching architecture. This modular approach not only elevates the system’s scalability and maintainability but also promotes agile development and deployment, a critical aspect in the rapidly evolving digital cash sector.

This unique combination of SOA and Microservices renders the Intelligent Daemon System highly adaptable, catering to a diverse range of consumer needs. It is well-suited for individual users who engage in straightforward Bitcoin transactions, as well as larger entities requiring more complex financial services and applications. The architecture’s modular nature allows for customisation and scalability in accordance with consumer demands, ensuring the system remains practical and relevant in the dynamic digital cash landscape.

The microservices approach in the Intelligent Daemon System enables dynamic scaling through its architecture, where each microservice functions as an independent unit. This independence allows for individual components to be scaled up or down based on the specific demands they encounter. For instance, if a particular aspect of the system, such as transaction processing, experiences a surge in demand, only the microservices handling these transactions can be scaled without affecting the rest of the system. This targeted scalability ensures efficient resource usage and maintains optimal system performance even under varying loads. Furthermore, the microservices architecture facilitates rapid deployment and updates, enabling the system to adapt quickly to changing requirements or emerging technologies in the electronic cash domain.

3. Subsystem Analyses

– Single-sig Transaction Management

The analysis of Single-sig Transaction Management in the context of Bitcoin wallets and transactions is intricate and involves several key aspects. Firstly, it focuses on simplifying the transaction process, ensuring that transactions are executed swiftly and securely. This management system is critical for maintaining the integrity of Bitcoin transactions, as it involves verifying transaction authenticity and ensuring non-repudiation.

 

This management aspect also includes the maintenance of the blockchain ledger, keeping it updated and synchronised across the network. The single-sig (single signature) aspect signifies that each transaction is authenticated by only one digital signature belonging to the owner of the Bitcoin wallet. This simplicity is fundamental in maintaining the system’s efficiency and user-friendliness.

Additionally, the system plays a vital role in the security of Bitcoin wallets. Requiring a single signature for transaction authorisation strikes a balance between security and ease of use. However, this also implies a heightened responsibility for the user in securing their private keys, as the loss or theft of the key could result in the compromise of their wallet.

Accounting, Bank, Exchange, Message Transaction Management

The subsystems of Accounting, Bank, Exchange, and Message Transaction Management in the Intelligent Daemon System play critical roles in ensuring the comprehensive functionality of the system.

• Accounting Subsystem: This subsystem is responsible for accurately recording and managing financial transactions within the system. It tracks and maintains the ledger of transactions, ensuring that all entries are correctly documented and reconciled.
• Bank Subsystem: This component focuses on managing user balances and facilitating fund transfers. It mimics traditional banking operations within the digital realm, ensuring secure and efficient handling of Bitcoin assets.
• Exchange Subsystem: This part of the system allows for the conversion between Bitcoin and other currencies. It supports trading activities and manages the complexities associated with currency exchange rates and transaction settlements.
• Message Transaction Management: This subsystem is pivotal in handling communication and data transmission within the system. It ensures secure and efficient messaging, which is crucial for coordinating transactions and maintaining system integrity.

The integration of these subsystems within the overall architecture of the Intelligent Daemon System is designed to provide a seamless, efficient, and secure experience for users engaging in Bitcoin transactions and management. Each subsystem works in harmony with the others yet retains distinct functionalities, contributing to the system’s robustness and adaptability.

Contracts Management

The Contracts Management subsystem in the Intelligent Daemon System is instrumental in facilitating smart contract applications. Smart contracts, being self-executing contracts with the terms of the agreement directly written into code, offer a significant advantage in automating and securing contractual processes. This subsystem is designed to handle the creation, execution, and management of these contracts within the Bitcoin environment.

It enables users to engage in complex financial agreements and transactions that are automatically executed upon meeting predefined conditions, thus reducing the need for intermediaries and enhancing the efficiency and security of contractual dealings. The integration of smart contract functionality within the system represents a significant step forward in the application of Bitcoin technology, expanding its utility beyond simple transactional capabilities.

Monitoring System

The Monitoring System in the Intelligent Daemon System is dedicated to overseeing the blockchain network. This system is crucial for ensuring the integrity and security of the blockchain, as it continuously monitors transactions and blocks for any anomalies or malicious activities. Key components of this system include real-time transaction tracking, block validation processes, and network health checks. These elements work together to maintain a secure and stable blockchain environment, crucial for the reliable operation of Bitcoin transactions and smart contracts. This monitoring capability is fundamental in detecting and addressing potential security threats, thereby safeguarding the entire system.

4. Daemon Core System

The Daemon Core System in the Intelligent Daemon System is a complex and critical component responsible for coordinating network processes. Its architecture is meticulously designed to ensure seamless integration and efficient management of various subsystems within the network.

The core system operates as the central command unit, managing communications between different modules like the transaction management, accounting, and contracts subsystems. It ensures these components work in harmony, providing a cohesive environment for transaction processing and smart contract execution.

One of the critical functionalities of the Daemon Core System is its ability to handle high volumes of transactions with precision and speed. This is essential in maintaining the performance and reliability of Bitcoin transactions. The system’s architecture is optimized for this purpose, employing advanced algorithms and data structures to manage transaction queues and processing efficiently.

Furthermore, the Daemon Core System plays a vital role in maintaining the security and integrity of the network. It employs robust encryption methods and security protocols to safeguard against unauthorized access and potential security threats. This is crucial in a decentralized environment like Bitcoin, where security and trust are paramount.

Overall, the Daemon Core System’s structure and functionalities are integral to the Intelligent Daemon System. It exemplifies advanced software engineering principles, ensuring that the system remains not only functional and efficient but also secure and adaptable to the evolving needs of the Bitcoin network.

5. Shared Libraries and Technologies

Common Ware API (CW API): This library provides essential services such as database connections, logging mechanisms, and general utilities. It forms the backbone of all subsystems, ensuring consistent and reliable interactions with various components of the IntDS.

Shamir’s Secret Sharing Scheme API (4S API): A specialized API implementing Shamir’s Secret Sharing, a cryptographic algorithm. This API plays a critical role in the secure distribution and reconstruction of sensitive information, crucial for the system’s security aspects, especially in key management.

Elliptic Curve Digital Signature Algorithm API (ECDSA API): This API is vital for generating public and private key pairs. Given the importance of secure transactions in Bitcoin systems, the ECDSA API offers robust cryptographic capabilities, ensuring the integrity and authenticity of transactions.

Mnemonic Code Generator API (MCG API): This API generates mnemonic codes, which are essential for creating human-readable representations of complex cryptographic keys. These mnemonic codes enhance the usability and accessibility of the system for users managing their Bitcoin wallets and transactions.

In addition to these APIs, the IntDS employs a sophisticated technology stack that includes Java, the Java Development Kit, JAX-RS for RESTful services, Jersey, Apache Ant, Groovy, PostgreSQL, and other critical software components. This stack ensures that the system is built on a solid foundation of reliable and widely adopted technologies, guaranteeing high performance, scalability, and adaptability in various operating environments.

The integration of these shared libraries and technologies demonstrates the system’s commitment to employing advanced, efficient solutions to address the complexities of Bitcoin transaction management. By leveraging these shared resources, the Intelligent Daemon System ensures a seamless, secure, and efficient operation, catering to the diverse needs of its users in the cryptocurrency domain.

The interaction of SigHash types with the system is detailed in Appendix K. SigHash types are crucial for defining how parts of a transaction are signed, affecting both the security and flexibility of transactions. The document outlines four SigHash types:

1. SIGHASH_ALL (0x00000001): This default type signs all inputs and outputs, protecting everything in the transaction except the signature scripts from modification. It is the most common form, ensuring that the transaction cannot be altered without invalidating the signatures.
2. SIGHASH_NONE (0x00000002): This type signs all of the inputs but none of the outputs. This allows anyone to change where the satoshis (Bitcoin units) are going unless other signatures using different signature hash flags protect the outputs. It offers flexibility in how outputs are handled but less security for the transaction recipients.
3. SIGHASH_SINGLE (0x00000003): With this type, only the output corresponding to this input (having the same output index number) is signed. This ensures that no one can change the sender’s part of the transaction but allows other signers to change their parts. However, the corresponding output must exist, or the security scheme fails. Only this input and other inputs are included in the signature, and the sequence numbers of other inputs can be updated.
4. SIGHASH_ANYONECANPAY (0x00000080): This type is unique. The input vector of the transaction copy (txCopy) is resized to one, and the subscript (led by its length as a var-integer encoded) is set as the only member of this vector. This type is like saying, “Let other people add inputs to this transaction; I don’t care where the rest of the bitcoins come from.” It allows for a more collaborative approach to funding transactions.

Each SigHash type offers different levels of security and flexibility, allowing the Intelligent Daemon System to handle various transaction scenarios efficiently and securely.

6. Security and Key Management

The Intelligent Daemon System (IntDS) places a high priority on security and key management, employing a robust framework to safeguard electronic cash transactions. The system integrates multiple layers of security measures to protect against various types of vulnerabilities and threats.

Elliptic Curve Digital Signature Algorithm (ECDSA): Central to the system’s security, the ECDSA is used to generate cryptographic key pairs, which are essential for authenticating transactions. This algorithm ensures the integrity and non-repudiation of transactions, a critical feature for electronic cash systems.

Key Management Strategies: IntDS implements advanced key management strategies, including secure key generation, storage, and recovery processes. These strategies are designed to prevent unauthorised access and ensure that critical materials are handled securely throughout their lifecycle.

Encryption Techniques: The system employs various encryption techniques to secure data in transit and at rest. This includes the use of industry-standard protocols and algorithms to encrypt sensitive information, providing an additional layer of security against data breaches and cyber-attacks.

Regular Security Audits and Updates: To maintain the highest level of security, the system would undergo regular security audits. These audits help identify and rectify potential vulnerabilities, ensuring the system is resilient against evolving threats.

The security framework of the IntDS is comprehensive, incorporating cutting-edge cryptographic algorithms, stringent key management practices, and ongoing security assessments. This multi-faceted approach to security ensures the safe handling of electronic cash transactions, instilling trust and confidence in the system’s users.

7. Challenges and Opportunities

Implementing the Intelligent Daemon System for electronic cash transactions poses several challenges while also offering unique opportunities in the blockchain space.

Challenges:

1. Scalability: Ensuring the system can handle increasing transaction volumes without compromising performance.
2. Security: Continuously safeguarding against evolving cyber threats and maintaining robust encryption standards.
3. User Adoption: Overcoming reluctance from traditional financial users and encouraging adoption among a broader user base.
4. Regulatory Compliance: Navigating varying and evolving regulatory landscapes across different regions.

Opportunities:

1. Innovation in Electronic Cash Systems: Pioneering new solutions and improvements in the electronic cash transaction process.
2. Enhanced Security Features: Setting new standards in transaction security, potentially influencing broader industry practices.
3. Market Expansion: Tapping into emerging markets and sectors that are beginning to explore blockchain technologies.
4. Collaborations and Partnerships: Forming strategic alliances with other technology providers and financial institutions.

Overall, while the challenges are significant, they are matched by the potential for groundbreaking advancements and growth in the blockchain domain.

8. Future Prospects and Conclusion

Looking back from 2024, the Intelligent Daemon System has evolved significantly since 2015. The advancements in its architecture reflect a deep engagement with emerging technologies and an adaptive approach to the ever-changing landscape of electronic cash and blockchain technology. This evolution, while rooted in past research and development, points towards a future rich with potential. The system’s journey from its initial stages in 2015 to its current state provides valuable insights into its trajectory, highlighting its role in shaping the future of electronic cash systems and blockchain technology. The progress made over the past eight years demonstrates not just technological advancement but a commitment to innovation and excellence in the field.

I hope you enjoy this book into the history of a project that has taken over a decade of work and the efforts of hundreds of individuals. I am proud to be launching the results that have come from research following this global project in England. While the eight years have led to continuous change through the system and the design has grown, the attached architecture enabled transaction volumes of over 100,000 transactions per second (TPS) in 2015. The system we are launching now will scale first to 1.5 million transactions per second and then grow with hardware-based needs to any limit.

Find the original documentation of the Intelligent Daemon System (iDaemon) here.