Bitcoin Unlimited Code Changes
This week, two large projects were submitted to the bitcoin Unlimited project’s “Dev” branch for review. Both of these projects were submitted in multiple Pull Requests. Of the two, developer Justaphf submitted a very large series of commits representing the past several months of his work. As his project’s commits were submitted in different Pulls, this section shall document the breadth of his changes while remaining as succinct as possible:
gandrewstone merged 17 commits by Justaphf from Pull Requests “#616” (Code for #616), “#617” (Code for #617), “#618” (Code for #618), and “#619” (Code for #619) into “BitcoinUnlimited:dev”. During these merges, 39 files were changed for a total of 1173 additions and 668 deletions. These changes deal with the creation of a Disk Operating System manager class that encapsulates all of the Disk Operating System related functionality of the client. These changes begin with Pull Request #616, which focuses on the refactoring of the different Disk Operating System structures of the Unlimited client. #616 moves several pertinent files and features from “main.cpp” into “nodestate.h|.cpp.” and adds “nodestate.h|.cpp” to the clang format list. The next set of changes in Pull #617 build directly on top of #616. The changes in #617 also deal with refactoring and are focused upon moving “CBanEntry” and “banmap_t” from “net.h|.cpp” into “banentry.h|.cpp”. Much like its prior commit, #617 also includes adding “banentry.h|.cpp” to the clang format list. The next Pull Request #618 continues the refactoring, and is built directly on top of Pull Request #617. This Pull deal with the refactoring of “CBanDB” and moves it from “net.h|.cpp” into “bandb.h|.cpp” . The changes in Pull #618 also contain the addition of a new helper function titled “GetDatabasePath” which will assist with unit tests, as well as adding both “bandb.h|.cpp” and “bandb_tests.cpp” to the clang format list. These three refactoring Pull Requests all lay the structure for a Disk Operating System manager class by making the refactored components only included where specified, which becomes the manager. The final Pull Request of this project #619 brings the initial three together and creates the Disk Operating System manager class itself. This Pull is built on top of Pull Request #619. This final batch of changes moves all Whitelist, Banlist, and Misbehaving behavior from “CNode” and “main.h|.cpp” to the manager. In addition, this Pull adds unit tests focused on Ban functionality to the manager which expands on the already existing “DoS_tests.cpp” unit test folder. These changes close by adding the manager files “dosman.h|.cpp” to the clang format list. Author Justaphf closed his final Pull Request’s comments with a short list of work that is upcoming that pairs with the creation of the manager class. This work still in progress will further organize the different Disk Operating System processes into the manager and clean up the code base. The changes within these 4 Pull Requests bring a new structure to the Disk Operating System processes of the client and significantly refine their structure. Usage of these processes is now easier and more streamlined lending greater refinement and performance to the Unlimited client as a whole.
The second of the two larger projects submitted to bitcoin Unlimited project this week was authored by developer kyuupichan. This project, similar to the prior project by Justaphf, is focused on encapsulating a specific element of the client and was submitted in multiple Pull Requests:
gandrewstone merged 2 commits by kyuupichan from Pull Requests “#640” (Code for #640), and “#644” (Code for #644) into “BitcoinUnlimited:dev”. During the merges, 15 files were changed for a total of 132 additions and 184 deletions. These changes deal with better encapsulation of the “CParallelValidation” system and it’s components. In the development process of a bitcoin client, it is important to ensure that the different systems governing specific tasks have adequate and easy to navigate organization and access. The changes in these Pull Requests address these house keeping necessities in relation to the “Parallel Validation” system. The first set of changes begin by creating a global unique pointer object specific for Parallel Validation. A unique pointer object in C++ code is an object that is capable of taking control of a specified pointer and managing it until it is deleted. In reference to the future opportunities that the creation of this “unique_ptr” object for Parallel Validation enables, kyuupichan says: “This gives us more control over the timing of construction. This allows us to do more initialization in the constructor and allows us to encapsulate other globals (more to come)”. Following the creation of this unique pointer object, the processes of “nScriptCheckThreads” and “allScriptCheckQueues” are now encapsulated within “CParallelValidation”. Furthermore, the creation of this object, in the developers own words: “permits making mapBlockValidationThreads private rather than public”. The second set of changes builds directly on top of the work outlined above, and is a simple refactor. The code for the system “semPV “ was migrated into “CParallelValidator” for better encapsulation. The remaining code present in “cs_semPV” was deleted following the move as it no longer served a purpose. The changes within these two Pull Requests provides a higher degree of organization and infrastructure to the Parallel Validation system. Future work and development for this feature will now be more streamlined and efficient while requiring less overall effort.
Bitcoin XT Code Changes
Of the commits merged into the code bade of the XT client this week, the most notable with the highest density of modifications is a Pull Request submitted by dagurval. The original commit of these changes submitted to the XT client last week was reported on in a previous code report, viewable here.
Bitcoin BTC1 Code Changes
In other code related news, Jeff Garzik, (developer handle jgarzik), is also spear heading the code development for Segwit2x. For readers unaware, a Consensus meeting was held in New York wherein a vast majority of the network hashrate came to a mutual agreement upon the grounds of scaling. This agreement stipulates that Segwit (Segregated Witness) and a hard fork of the blockchain to 2mb capacity blocks will be coded and implemented by the community. This project, dubbed Segwit2x, has been in development for almost 2 weeks now following the New York Consensus meeting which ended on May 25th, 2017. More information about the agreement and the supporters who signed it can be read about here. For informational purposes, major additions to the Segwit2x project, (Stored in the BTC1 Repository), will be henceforth included in these reports under their own section header.
Bitcoin Core Code Changes
iaanwj merged 2 of his own commits from “laanwj:2017_05_peer_listenaddr” into “bitcoin:master”. During the merge, 6 files were changed for a total of 53 additions and 13 deletions. These changes add the “Listen Address” functionality to the incoming connections in “getpeerinfo”. The changes in this Pull are fairly straightforward. Developer iaanwj describes his changes very similar to what is stated above: “This adds the listening address on which incoming connections were received to the CNode and CNodeStats structures.” The resulting listening address is then reported in “getpeerinfo” per connection. This modification has several benefits for the client. In the developers own words: “This can be useful for distinguishing connections received on different listening ports (e.g. when using a different listening port for Tor hidden service connections) or different networks.” Adding the listening address to the incoming connections listed in “getpeerfinfo” gives the client a greater degree of control and flexibility between listening ports and allows for an easier distinction between them.
MarcoFalke merged 5 commits by jnewbery from “jnewbery:shared_util_function_test_config” into “bitcoin:master”. During the merge, 9 files were changed for a total of 157 additions and 175 deletions. These changes modify the configuration files utilized by the functional and utility tests of the client. Previously, these two test classes would both require a configuration file generated by “./configure” to run. Instead of requiring the “./configure” system to produce a unique configuration file for each test class, developer jnewbery modified the behavior of this system so that the 2 unique configuration files would become merged so they can work for either / both of the test cases. In the same vein of combining test elements, these changes further combines the tests “bctest.py” and “bitcoin-util-test.py” into the same module. In the developers own words: “bctest.py is only used as an import by bitcoin-util-test.py. There’s no value in keeping it as a separate module, so this PR merges them into a single module to keep building and packaging simpler.” These changes clean and organize these test systems allowing for quicker and more efficient run times. Furthermore, these changes allow for a greater ease of modification in the future due to this organization.
sipa merged 10 of his own commits from “sipa:fast_rand_tests” into “bitcoin:master”. During the merge, 26 files were changed for a total of 202 additions and 190 deletions. These changes modify the randomness used in unit testing within the client. As the bitcoin ecosystem is built upon a cryptographically secured currency, cryptography and associated randomness are common themes in the development world for bitcoin clients. The bitcoin Core development team has created and developed several different cryptography enhancements and libraries over the years, and one such project bearing their name is “FastRandomContext”. The “FastRandomContext” class was the focus of a few improvements several weeks prior as developer sipa modified it’s internal structure to employ ChaCha20 as means of generating randomness. This class is employed in unit testing within the client to generate random scenarios for the unit tests to struggle through greatly improving internal client vetting. In addition to “FastRandomContext”, some of the functions from the “Rand*()” system are also employed to further increase randomness. Developer sipa noted that this process was not as efficient as possible as the mixing of the randomness generators was causing inconsistencies. He also noted that the inefficiency would exacerbate as the “Rand*()” system will soon be forced to always employ very strong randomness which if combined with unit tests would slow them down further. The changes in this Pull Request address this inefficiency by forcing all unit tests to only receive randomness from the “FastRandomContext” class. Manager of the bitcoin Core project iaanwj weighed in positively on this decision, stating: “Concept ACK, also been meaning to do this. There is no need for strong randomness in tests and speed is very important.” These changes help to speed up all unit testing within the client while ensuring that proper levels of randomness are maintained for strict and strenuous testing.
The scaling debate is approaching its breaking point, and now more than ever, it is important for every individual, not just miners and software developers, to do their part. All readers are recommended to download and establish a full node supporting the client of their choice should they have the technological and economical means to do so. Download links for each of the major full node clients:
– Bitcoin Unlimited – (Pro immediate block size increase via hard fork, Emergent Consensus)
– Bitcoin Classic – (In line with the view points of bitcoin Unlimited, Emergent Consensus)
– Bitcoin XT – (In line with the view points of Classic and Unlimited)
– Bitcoin Core – (Block size increase slowly over several years, immediate soft fork w/ Segwit)
If any reader feels daunted, confused, or overwhelmed by this task, please do not hesitate to reach out