Starting at block 2,265,984 (~May 19th), MWEB will be activated on mainnet, and everyone who has updated to v0.21.2 will be able to start sending and receiving LTC using the MWEB.
Final v0.21.2 Release
The final release will be available later this evening at https://litecoin.org/. Anyone interested in using the MWEB, and especially those who installed one of the earlier release candidates, should upgrade to the official v0.21.2 before MWEB activates. If you wait until after MWEB activation to upgrade, you’ll unfortunately be required to resync the blockchain from scratch.
While the consensus logic has been solid for a while now, the wallet has undergone drastic changes these past few months as we worked to resolve issues found during testing. The major workflows all seem to be working well now, and I expect most people to be able to use MWEB without issue, but this was an enormous change, so don’t hesitate to report any bugs or behavior that seems off to you.
I was away last week celebrating my 10th wedding anniversary (yep, I’m ancient), so I’m even later than usual this month. Sorry about that!
May (and part of June) 2022 Progress:
Critical Wallet Bug
Shortly after MWEB went live, a few users reported not receiving coins sent to their MWEB address. After some investigation, it was found that old pre-HD wallets that were upgraded using the upgradewallet command were generating stealth addresses incorrectly. https://twitter.com/DavidBurkett38/status/1528575311335591937
Fortunately, all funds were safu, but a fix was needed in order for those users to recover their coins.
v0.21.2.1 Release
A new version of Litecoin Core (v0.21.2.1) was released with the fix. You can download it here.
If you’re one of the few unlucky individuals whose wallets are not receiving correctly, upgrade to the new version and then run rescanblockchain 2265984 from the Litecoin console to recover your coins.
Other than this wallet bug, MWEB seems to be working really well. If you have any issues though, please report them on github, or in the MWEB telegram channel.
Now that activation is behind us, it’s time to move to the next big task. Starting this week, we’ll begin working on the design for mobile wallets and other light clients. MWEB presents many technical challenges for light clients, so it may take some time before we settle on a design, but I’ll continue to keep you updated on the progress each month.
After a meeting with @coblee and @losh11, we agreed on a rough design for supporting light clients. The following is subject to change, but here’s my current thinking, taken from the LIP (Litecoin Improvement Proposal) I’m writing to help standardize the light client sync process:
1. Download and verify all headers for the longest chain.
Headers can be requested from peers using getheaders messages, which will be returned in headers messages using the process described here.
2. Download and verify the HogEx transaction and MWEB header for the most recent block.
This data can be requested using a getdata message with type MSG_MWEB_COMMITMENT, which will be returned in the following 3 messages:
merkleblock - Contains the hash of the HogEx transaction, and enough to validate that it is the correct transaction according to the block’s tx merkle root.
tx - The serialized HogEx transaction.
mwebheader - The serialized MWEB header.
The light client shall validate that the hash of the HogEx transaction in the tx message matches the hash in the merkleblock message, and that it’s the last transaction committed to by the merkle root of the block. It shall then validate that the pubkey script of the first output contains the HogAddr, which shall consist of <OP_8><0x20> followed by the 32-byte hash of the MWEB header. Finally, it shall validate that the blake3 hash of the MWEB header matches the hash contained in the HogAddr.
3. Download and verify the UTXO leafset bitmap.
The leafset can be requested using a getdata message with type MSG_MWEB_LEAFSET. Verify that the hash of the bitmap matches the leafset_root value in the MWEB header.
4. Download the compact UTXOs and PMMR parent hashes.
These can be requested piecemeal from multiple peers in parallel using getmwebutxos messages. As compact UTXOs are downloaded and verified to belong to the longest chain, wallets can check to see if they own the outputs using the process described in LIP-0004 (Output Identification). Any UTXOs determined to not belong to the wallet may simply be discarded.
NOTE: A compact UTXO is an unspent MWEB output sans the rangeproof. When designing MWEB, we chose to hash them in a way where we would only need the hash of the rangeproof to verify the output hash, meaning light wallets can avoid downloading the nearly 1KB rangeproof for each UTXO.
Once I’m finished with my first draft of the LIP, I’ll submit it for review and then start making the P2P protocol changes necessary to support the design.
Last month, I described the LIP for standardizing the light client sync process. The LIP is just about ready for review, and I’ve made good progress implementing the changes.
Here is the status of each step:
1. Download and verify all headers for the longest chain.
Light clients already download the header chain, so no changes are needed for this step.
2. Download and verify the HogEx transaction and MWEB header for the most recent block.
I’ve written the code for responding to an MSG_MWEB_COMMITMENT getdata request with a new message containing the Merkle Block, HogEx transaction, and MWEB header. Tests were also written and everything appears to be working, so it’s ready to be submitted for review.
3. Download and verify the UTXO leafset bitmap.
The new getdata message type was added, as well as a new message for responding with the actual leafset.
Peers will be able to request a leafset for a recent block, but it may not be the most recent block. We only keep one working version of the UTXO leafset, so if an older leafset is requested, we need to “rewind” the leafset to the block height it’s requested for. Currently, I’m working on the code for performing this rewind.
4. Download the compact UTXOs and PMMR parent hashes.
The changes for this step have not yet been started.
I intend to wrap up the LIP this week and get it submitted for review. I should also be able to submit a PR for steps 2 & hopefully 3 before next month’s update.
The first draft of LIP-0006, which details the changes and additions to the p2p protocol, is now out for review.
A pull request has also been submitted for downloading and verifying HogEx transactions, MWEB headers, and the UTXO leafset bitmap. This covers steps 2 and 3 of the light client sync process.
That means the only piece still missing that light clients need to be able to sync MWEB data is the ability to download the compact UTXOs and their merkle proofs. As documented in LIP-0006, we want light clients to be able to request UTXOs from peers in parallel using a getmwebutxos message, which will be returned in a mwebutxos message.
I should be able to get this last piece coded and out for review this month. After that we’ll look at creating a new release (v0.21.3) with these new changes, and then start working with third party wallet developers to hopefully get MWEB added to all major LTC wallets
The code for supporting retrieval of MWEB UTXOs (unspent coins) for use by MWEB light clients is out for review It was designed so that light clients can request UTXOs in batches, and verify that each batch is committed to by a recent block. This is similar to how they can currently retrieve regular LTC transactions and verify that the transactions are committed to in a block’s merkle root. In order to do this, each batch of UTXOs has to come with enough hashes to verify the outputs belong to the Merkle Mountain Range (MMR) that commits to the MWEB UTXO set.
This was more difficult to implement than I had hoped. Since MMRs are reasonably more complex than the simpler merkle roots that regular LTC blocks use, it turned out to be quite a challenge designing the algorithm that determines the minimum set of hashes necessary to verify the UTXOs. Because of the complexity of this algorithm, I’m going to hold off on creating the new release until some of the light client dev teams have had time to understand how the new code works. Instead, I’ll host a node with the latest code that wallet developers can develop and test against, and if we need to tweak it to make it a little simpler (at a slight efficiency cost), we can do that much easier than if the code was already released.
Aside from working with wallet developers and improving documentation to assist with their development, I’ll also be backfilling some functional tests and fixing a few latent wallet display bugs this month in preparation for when we decide to go forward with the v0.21.3 release.
I had a chance to talk to some wallet developers at the LTC Summit, which I used to better understand what each needs in order to support MWEB. From these discussions, it appears there will be a few different approaches taken for syncing and identifying MWEB wallet transactions, each with different performance, security, and privacy tradeoffs. My goal is to make sure we provide all of the APIs, p2p messages, and libraries necessary to give each wallet developer the tools they need to go their own way.
I wrote up a quick document briefly describing the two high-level approaches (client-side and server-side filtering) that wallet devs can take to support MWEB, and the pros and cons of each: MWEB Light Clients. I’ve been focusing strictly on client-side filtering these past few months, since that offers the strongest privacy and security guarantees, and while most wallet developers were on board with the idea, one showed more interest in server-side filtering for performance reasons.
View Keys
Server-side filtering of txs & outputs is done using “view keys,” which can be shared with a server so that they can identify transactions belonging to your wallet. Right now, that also means the server would learn the amounts of your transactions, unfortunately. There is a way this can be done without revealing the amounts by adding a new output format version, so this may be worth looking into in the future sometime. In the meantime, I’ve decided to work on adding full view key support to LTC Core that can serve as an example for other wallets.
This month, I plan to finish adding in view keys and get that out for review, so I can start working on MWEB support in our raw transactions APIs. These are the APIs that some third-party wallets, in particular hardware wallets, use to build transactions. We’ve already had a few 3rd party devs request MWEB functionality for these APIs, so it would be great to be able to include that in our next release.
I decided to take a short break from light client work, and jumped ahead a little bit to start adding MWEB components to the PSBT(Partially-Signed Bitcoin Transaction) format. PSBT (See BIP-0174 and BIP-0370) is a flexible transaction serialization format that can be used to support building transactions using advanced workflows such as sending from a multisig wallet, or in the case where most interested in: hardware wallets.
In order to add MWEB support to PSBTs, I first had to merge in PSBTv2 (BIP-0370), which can be seen here. I then defined the MWEB fields we needed to add, and added them to the PSBT data structures along with serialization and deserialization here.
Add functional test(s) covering the hardware wallet workflow
Document the PSBT format additions
View Keys
Since the hardware wallet PSBT workflow depends on view key support to identify UTXOs belonging to the wallet, I will continue working on view key support in parallel
I’ve been feeling under the weather for some time now, so I’m just now getting around to December’s update
December 2022 Progress:
I continued working on PSBT, and quickly discovered some limitations in my initial design. I had to redefine a lot of the MWEB fields before I could proceed with the PSBT tasks I listed last month. After doing that, I was able to complete the first 3:
We released a new minor version of Litecoin Core (v0.21.2.2) in February which contained some important security fixes. If you haven’t upgraded yet, you can download it here.
v24 Release
I’ve started working on a new major release for Litecoin Core that will contain everything that I’ve been working on this past year:
PSBTs - To encourage hardware support for MWEB
P2P Support for Light Clients - To allow for mobile and other more user-friendly wallets
View Keys - To support “watch-only” wallets that don’t require having access to private spend keys in order to identify transactions you’ve sent or received.
Payment Proofs - For traditional litecoin transactions, where proving you sent coins to an address could be done by just signing a message with your private key, and pointing to the transaction on the blockchain which shows the source address (which you just proved to be the owner of), and the destination address and amount. With MWEB, addresses and amounts are all encrypted, so in the case that you need to prove to an arbiter that you made a payment, a more complex proof is needed. This proof is described here.
Descriptor wallets - The wallet code, originally written by Satoshi, was very limited, and not designed to support the many script types and features that have since been developed (P2SH, Taproot, BIP32, and now MWEB). Developers have been forced to hack in new wallet features in a messy, unsustainable way. To deal with this, BTC developers introduced “descriptor wallets,” which provide a more extensible replacement for the existing wallet backend. You can read more about them here. With Litecoin Core v24, we’ll be enabling descriptor wallets, and extending them to handle MWEB addresses, which should help solve a number of limitations we had with our initial MWEB wallet design.
So far, I’ve added the classic (pre-MWEB) litecoin code changes to the v24 bitcoin code, and am working on fixing tests. I’ll then merge in the already-released MWEB code, followed by the new features listed above, submitting everything for code review after each step. This will be a lengthy process taking us well into the summer, but the end result should be a new release that’s up-to-date with bitcoin core, and will include all of the new MWEB features everyone has been waiting for.
Over the past couple of months, the focus has been primarily on integrating the classic (pre-MWEB) Litecoin code changes into the v24 Bitcoin code. This process is nearing its conclusion, with all unit-tests fixed, and just a few functional tests remaining to be fixed. Once the last few tests are resolved, I will submit the merged code for review as planned.
I’ll be updating the following task list each month as we work toward the release:
Integrate pre-MWEB litecoin into bitcoin’s v24 codebase (A few tests remaining, then review) Merge in the previously-released MWEB code (Not started, expected by end of May) P2P Support for Light Clients (Implemented, in need of merging & thorough review) Enable descriptor wallets w/ MWEB address support (Not started) Finish implementing PSBTs (Mostly implemented, needs testing and review) View key support (Rough design known, implementation started) Payment Proofs (Design outlined in LIP-0004, implementation not started) Release Notes (Not started) Gitian Build & Publish (Not started)
Progress continues with the integration of Litecoin code changes into the v24 Bitcoin codebase. The month of May saw a major milestone with the resolution of all remaining functional tests for the pre-MWEB merge task. The code has been submitted for review here.
Furthermore, we’ve made significant strides in merging the previously-released MWEB code. The majority of this task is now complete, marking another key step forward. However, there are still a few changes needed on the wallet side and several tests that need to be fixed to finalize this phase.
I also took some time to work through bitcoin’s descriptor wallet code, and have a decent understanding of how it all works, and how the data is stored in the wallet database. I’ve started experimenting with different ways of supporting MWEB keys and addresses using descriptors, but a lot more thought is needed before I commit to a design.
Here’s our updated task list as we move closer to the v24 release:
Integrate pre-MWEB litecoin into bitcoin’s v24 codebase (In review) Merge in the previously-released MWEB code (Majority merged, a few changes & tests left) P2P Support for Light Clients (Implemented, in need of merging & thorough review) Enable descriptor wallets w/ MWEB address support (Design in progress) Finish implementing PSBTs (Mostly implemented, needs testing and review) View key support (Rough design known, implementation started) Payment Proofs (Design outlined in LIP-0004, implementation not started) Release Notes (Not started) GUIX Build & Publish (Not started)
The initial MWEB code has been merged into the v24.x codebase, and will be submitted for review shortly. I encountered a couple of obstacles that slowed down momentum, particularly on the wallet side.
When MWEB was implemented, I was able to make its node logic very modular, making merging that relatively easy (with the exception of the mempool, which is still rather complex). I was not as lucky with the wallet code, however, which requires quite a bit of merging. And it turns out that Bitcoin’s wallet code has been refactored quite a bit over the past few releases, so this ended up being a much slower, more tedious process than I hoped.
Updated Task List:
Integrate pre-MWEB litecoin into bitcoin’s v24 codebase (In review) Merge in the previously-released MWEB code (Will update with review link shortly) P2P Support for Light Clients (Implemented, in need of merging & thorough review) Enable descriptor wallets w/ MWEB address support (Design in progress) Finish implementing PSBTs (Mostly implemented, needs testing and review) View key support (Rough design known, implementation started) Payment Proofs (Design outlined in LIP-0004, implementation not started) Release Notes (Not started) Gitian Build & Publish (Not started)
My goal for July is to implement MWEB address support for descriptor wallets.
Integrate pre-MWEB litecoin into bitcoin’s v24 codebase (A few tests remaining, then review)
Integrate pre-MWEB litecoin into bitcoin’s v24 codebase (