Scaling Bitcoin workshop : Milan 2016

Build scale operate

Build - Scale - Operate: The Three Pillars of the Bitcoin Community

Meltem Demirors

Eric Lombrozo

https://twitter.com/kanzure/status/784713580713246720

video: https://www.youtube.com/watch?v=8BLWUUPfh2Q&t=2h29m19

Good morning. This is such a beautiful conference. Eric and I are excited to speak with you. We are going to outline what our goals are. We want to be clear that we're not here to define vision. Rather, we're here about how to grow vision around how to grow bitcoin from a technical community perspectives. We're going to share some of our challenges with bitcoin. Eric from the development perspective, and myslef from the business perspective. I want to offer some perspectives from ourselves and others to grow bitcoin community. We have setup an anonymous google doc. If oyu disagree, it's just bit.ly/scalingideas and then the goal is to - we want to leave this conference with shared next steps as a community, not just for people who have been part of the community for the past 4 or 5 years, but also some of the new members who want to participate.

There are many different stakeholders in the bitcoin ecosystem. It's fragmented. We all have different people we talk with. Lots of loosely-coupled organizations. Some are grass-roots and focusing ob psecific objectives, while others are more general to the ecosystem. This conference gives everyone an opportunity to come together and talk about building a shared vision.

What we were thinking is that there are three key stakeholders. On the build side, there are Core devs and Core contributors of course, but also CTOs and engineers building products on bitcoin. Academia is driving research whether it's ways to apply new ways to apply to bitcoin. Business people are building as well; products. On the scale side, there's a lot of vision that goes into this, and our community is lucky to have this. There are also PR departments and marketing that spread the message about what bitcoin is doing. For bitcoin to work, there are miners and exchanges to move money around. There are a lot of different collaboration. We want companies to work together in a collaborative way.

The challenge is that-- we're trying to high-level outline what these are. There are inherent complexity to bitcoin protocol, which has dependencies that cannot be resolved elegantly. Eric will talk about that. There are also accidental complexities due to the way the project was built, as an experiment, and now $10 billion dollars of value has been put into this network that has been built. There are real issues about maintaining compatibility. The third and maybe controversial I think there's inadequate and methods of and techniques-- the method of development, it's hard to get experience, and meaningful contribution requires high skill. There's a lot of complexity in how we work together. What does that mean in terms of growth? We're constatly re-creating and re-inventing, what if we shared what we already buitl, and build optimal solutions -- perhaps we could have more progress. Can we stop focusing on problems that have already been solved? And maybe we can start thinking about how to coordinate more so that more features can be built, rather than just build priod?

Can I hand it over to Eric, who can talk about contributing to Bitcoin, and I'll talk about operating. And then we can share some suggestions.

Thank you Meltem. So when I first started contributing to Bitcoin 2011, I started writing my first bitcoin applications. I started the same way a lot of other people probably do. I started with the RPC build into Bitcoin Core. This was RPC was not designed for industrial use. It allows oyu to build some simple applications, but it's not the same thing we would need to build out an entire industry. I wa able to build a couple applications just to see how bitcoi nworked. I ran into limitations and then dug deeper... the only thing you could really do is get into the wire protocol itself of the bitcoin p2p protocol. I had to write several libraries to write some applications on top of this. This is pretty low level. Most application developers probaly don't want to go this in depth. While there are some libraries that abstract some of this stuff for you, you still need to have an in-depth knowledge of the low-level stuff. You need to be able to query what you need to query, without running a full node.

So this is the first pull request that I tried to submit and as you can see, I was very excited about getting this great multiple wallet support complete. This was another way you can actually contribute-- you could try to incorporate them into the Bitcoin Core software itself. After the obligatory bikeshedding and nitpicking and the rebases and doing that over and over again for several months... someone said "This is awesome work and I hope it makes it to btc soon". Unfortunately it didn't work that way; Wladimir closed it, which I don't blame him for, it got too out of sync for the project. I don't blame him for that. I'm sure that other people have had this frustration as well. I think people get upset-- I thikn they are right to be upset thhat it's frustrating to contribute to bitcoin. The process does have some serious bottlenecks. Not many people can do the code review, and there's a lot of risk inherent in changing the code.

I think something that could help fix this to a large extent and reduce the problem is using layered architecture. In the case of the internet, you have a base layer like IPv4/IPv6, which just gets a packet from point A to point B, and then an application layer that interprets data and shows it to the user and has some interaction or whatever. Bitcoin has a similar structure that we can think of. At the base layer, we have consensus, and then as you go further up, the compatibility requirements become less strict. Applications might want to share data, but it's not critical. At the consensus layer, if two nodes disagree, then you end up with mutually incompatible histories and that's something we're trying to avoid.

One thing I've been trying to work on is bip123, which tries to make it so that bip authors when they submit a BIP they categorize their bips according to layers and makes it easier to sort through them and to prioritize them and be able to see which ones are more critical for review. For the application layer, you might not need any Core developers to review it. Just build an app, great. At the consensus layer, it requires the entire community to agree. In this particular BIP, I separated the BIPs into 4 different categories. You have wallet- like address formats, bip32 you have password-encrypted wallet stuff, tihngs that are necessary to have compatibility between different applications but not necessary for the entire ecosystem to function. At the RPC level you have interfaces that applications might depend on to interface with the bitcoin network. You can have several different APIs and several different models . At the peer services layer, you need some level of -- they need to agree on the data format and you can add new messages and you can deprecate old messages. In the consensus layer, you need everyone to agree, and you must try to keep from making incompatible changes. Consensus rule changes are one of the most difficult challenges. This requires everyone agree. It's really hard to get everyone to agree on something. Engineering always has tradeoffs. There might be some reason why someone doesn't want something. It makes it very difficult. The fact that nobody can change the rules from under you means that you have certain guarantees that your coins have value and that the rules are encoded there so that nobody can change it from under you. This is frustrating to people who want to innovate on the protocol because it makes it difficult to change things. There are mistakes that we can see now, that in hindsight we would have changed the protocol in the first place, but it's difficult to coordinate this change now.

Satoshi made this post in 2010 where he says and I quote, once version 0.01 is released, the core designed is set in stone for the rest of its lifetime. This is basically Satoshi saying that changing things is difficult. He also said that a second incompatible version would be a massive headache for him. Any incompatibilities would cause serious problems for the network. Fortunately we have made some progress since those days, or start to do those things without those risks. Hard-forks are one of the most naieve ways to change the consensus rules. If you think transaction formats are more compact or whatever, well guess what, not all clients are going to accept that. Basically you're going to have nodes that don't agree, and you're going to get a hard-fork. Soft-forks are the best mechanism we have so far for deploying consensus changes because not everyone needs to upgrade at the same time. You just need a supermajority of miners to upgrade first, and then the rest of the netowrk can upgrade at their own leisure. I don't think this is something that Satoshi considered, but right now it's basically the only thing used to deploy consensus changes.

We need to think about bootstrapping networks. We're trying to upgrade the system while it's moving and there's billions of dollars of other people's money on this, and if you screw up then people can lose a lot of money. This is not something we take lightly. It's very hard to do this. This has been a big problem for being able to innovate at that level. We have the development bottlenecks which have to do a lot with the fact that we have a limited number of code reviewers and they all have to look at the same stuff. No matter how risky the code is, it usually has to change consensus code or something, so that makes it a problem. We need a different modularization where we can have different modules working in loose coordination rather than hard coordination. We need better--- we're working on BIP mechanisms to do soft-forks; there needs to be more research for cross-chain or atomic swaps or other mechanisms for bootstrapping this system. We need to be able to do this without breaking anything, of course.

How was the node architecture setup in the original client? You had this monolithic system which is Bitcoin Core, and then you had RPC and p2p protocol layer. Anything you want to change in Bitcoin Core area requires you to go through main.cpp and going through the code review process and merge process and that can be pretty frustrating. We've already pulled out mining from Bitcoin Core and a lot of the wallets have been pulled out. We can pull out the database, and we can have a different levels of query depending on which applications you want to support, to do just basic block validation and relay you don't need address indices or transaction indices but some applications might require that. Instead of forcing that into Bitcoin Core itself, and a lot of people quite rightfully think that Bitcoin Core should not be made into a huge application that does everything; if you could separate that out and make it a shared service, then other applications could use it, and we would solve the problem of people not having a good way to make applications.

We could break out the networking part, and then we could split out libconsensus and a library that does just validation and we can specify policies independently and all this other neat stuff. Once these aspects are separated with better interfaces, we could have many more teams working in parallel without needing close coordination on everything. Now I am going to hand it over to Meltem.

Alright, so, ... I think what's interesting from my perspective, I'm not a developer and I am not in that part of the community, I work with businesses built on top of bitcoin. A lot of the companies built in 2013-2015, their business models are dependent on the use of bitcoin increasing and the adoption of bitcoin increasing. So we have this money being poured into the funnel at the investors and the opportunists and people intellectually curious about bitcoin, but the Core group of people interacting with bitcoin has not changed much in the last 3 years. Why is that? And how do we get the ecosystem to become much bigger?

Everyone has seen this "innovation curve". It's not that exciting. But what I think is interesting is that-- some people say we're still in the early adopter phase, it's limited but there will be a tipping point and then lots of people will be in bitcoin and it will be great. I don't think tihs is true necessarily. I think bitcoin has reached the majority in its current form, and the current technical capabilities of bitcoin need to change... building with bitcoin today is really hard. This room is probably 90% of the talent that could do that feasibility. So that's not really scalable particularly if you're talking about taking this technology and making it broadly accessible.

So who cares about Bitcoin today? The way we came up with this chart was that several months ago when the scaling debate was raging back in March, we weere having interesting internal conversnations like why is this happening, who are the people are upset and who do we need to talk with? From my perspective, who has the actual ability to influence what happens in Bitcoin development? Who is actually interested in it? We start to see today- there's a bit of a mismatch. There's developers building Bitcoin and they have a high interest. Miners as well have a large incentive to be involved in bitcoin. You go down a few layers and think about people building on top of bitcoin, they might not have any ability to influence the development process. Same thing with corporates. They might have some interest, but their role is limited. The really interesting piece is the holders or observers who are on forums or in our community but not part of the group influencing development. They care a lot about what happens in bitcoin but they have no ability to influence deveopment. There's not that many roles in our community. There's people who feel passionate about bitcoin but they have no way than expressing it other than expressing outrage on forums or creating new forums or starting their own initiatives that work counter to Bitcoin's core group. I don't think the general public matters for bitcoin, so I'm leaving them out.

Okay if we've reached 100% adoption by all potential adopters of bitcoin in its current form, then we've exhausted this curve over time. 2016 has been 8 years. We've built bitcoin, we've made it work, we made it better there's ways to get money into and out of bitcoin, there's a bunch of applications that allow bitcoin access, but now we have to figure out how to scale it out and we haven't figured that part out, which is why this workshop is called Scaling Bitcoin.

So why does this matter? If we have reached penetration in this first market, then what is the next target market? Who is the next group of people and what do they need from a technical perspective? We can only grow if we communicate with the consumers of bitcoin and keep that in mind during the development process. Where would these people fall on this spectrum if there was a new feature set in bitcoin? Bitcoin Core figures out what features to implement so that more people can build and use and access bitcoin. As you think about stakeholders, where are ... what are all the ways that technology innovation needs to happen in bitcoin so that it can scale and reach more markets? So this is hard.

Everyone in this picture has different incentives, but we all have to work together because we all rely on the same core technology. How do you do that? You can have events like these; forums; private meetings. But how do you scale that? One-on-one conversations are really hard to scale. If people disagree, I'm happy to talk afterwards. I think bitcoin is not just technical software but it's social software. Well how does the group of bitcoin work?

There's this paradox of group think "hey this is good and it has to be protected because it's Bitcoin". I think we look at external enemies as being a unifying cause; so we find group cohesion about who is the good guys and who is the bad guys. It's not very cocnstructive. It pits each other against each other. There's also this religious aspect whether it's Satoshi or ways that the protocol works. This leads to resistance about ways to grow bitcoin. If people disagree, then I'm happy to talk about this. I think the problem is that- and this is not the fault of any one person it's just how things happened-- Satoshi built Bitcoin and he had one vision and he assumed a certain behavior. It was more of an experiment. Users came on, and they had different behavior. And then people ran the system and they realized the technical and social issues couldn't actually be uncoupled. They were intertwined in a strange inseparable ways. And the conversational context around Bitcoin, there was this small Core group that was planning about the future of bitcoin did not scale to the much larger group. So the larger group really felt ownership. How do you scale beyond that small group?

I like frameworks. I'm sorry. It's probably boring. So how do you scale beyond that small group and make it bigger? I think we have talked about this at other scaling conferences. We have this momentum now around bitcoin and there's a great opportunity to work as a community to grow. Awareness of bitcoin and it's -- knowledge is distributed on slack, IRC, it's not organized, someone should organize the knowledge. Someone should do this. A lot of people view bitcoin very negatively; we haven't done a great job shaping the narrative. People have a lot of assumptions about bitcoin that are really wrong at a fundamental level. So how do you start address that spread of misinformation about what bitcoin is and what bitcoin is capable of today?

Adoption-- the decision about whether to adopt bitcoin, features, user design, experience, these are really important. Bitcoin sucks compared to using venmo, or a bank, or an internet app on your phone. Bitcoin is not that great. You have a long address, it's scary sending your first transaction in my first transaction I checked my address maybe 30 times. I was terrified. This is what the average person thinks about. It's really hard to adopt bitcoin because it's not inherently user-friendly but also at the technology level and innovation level it's really hard to adopt. Implementing bitcoin is even harder. If you want to link bitcoin, it's really impossible unless you have Eric or one of the developers who knows how to go under the hood of bitcoin and actually make it work.

Comparing bitcoin against other technology choices, and determining whether it's efficient or elegant, I think academia has a lot to offer here. So I want to focus on just these three. In attitudes, there's a lot of misunderstanding. I work on the business side. A lot of people ask me about using bitcoin, and a lot of banks have people working on bitcoin and who want to use bitcoin and play with it but there's no economic data available for modeling the way that people make business decisions is based on opportunity data. So if you can't size the statistics, then you can't make a business case. A lot of people who participate or drive value, either economic or social value from hte bitcoin ecosystem, have not been required to participate. As you think about the work in the Bitcoin Core development, there's just a handful of organizations hwo support that project. Why is that? That's a really big problem.

The feature set of bitcoin is limited, which limits use cases. There are limited resources, lots of dependencies. We need people to contribute more resources. We need to start resolving dependencies. We need feedback loops on the feature set. Who are the customers of bitcoin? What do they want? How do they prioritize features? I think that dialogue is just now starting. I hope to see that evolve in this room and beyond.

Implementing bitcoin-- it sucks. It's hard to find talent. It's hard to understand how to get bitcoin into another system, also from an architectural perspective. Does it make things more efficient, or is it just interesting? And what about reinventing the wheel here? A lot of companies will build their own application stacks, when they could just existing technology stacks that owrk really well at doing certain things. There's opportunity here to leverage work and standards already created rather than reinventing and rebuilding the same things.

Project ideas

We had some ideas for projects we could all work on together. Eric will outline some first, and then I'll outline some.

Working on Bitcoin Core for the past several years, it's clear that we need to resolve these bottlenecks. I think we can create a much better set of tools and much btter framework for projects. I think separating the dependencies and designing interfaces to decouple them. Refactoring them into individual units will allow us to do this in a way where we don't have to do everything at once, we can do it incrementally, and get us to our goals incrementally.

On bitcoin applications, extending research and the body of knowledge around non-technical issues would be useful. In this conference, we're focusing on technical issues. But how do we start engaging people outside of the development community? How do we think about building infrastructure whether it's exchanges or wallets or other parts of the ecosystem? How do we figure out an efficient way of working together? How do we manage technical projects across the community? This is not the core protocol layer but rather the application layer. More coordination on the application layer will make it easier for people outside of the bitcoin community to come in and use bitcoin. I thikn the last piece- a request I hear all the time is a reference architecture for how bitcoin will fit into enterprise situations or app architecture. So let's say I have learned to love bitcoin, that's a big hurdle that requires a lot of knowledge. But then how do I implement and adopt bitcoin? There's a lot to be done there not so much for Core development but also for enterprise integrations and so on.

Q&A

Q: Why do you think that the VC-funded companies are not funding Bitcoin Core development?

A: I think it's inherently difficult to contribute to bitcoin without some very special skillsets. A lot of ventur capital backed projects hire developers that have more typical skillsets for what they are looking for. Unfortunately the way that bitcoin works right now doesn't allow the level of control and sophistication in applicaiton development without those special skillset. I think these companies are working on top of infrastructure development. I think they ad hoc their own development backend that kind of works but it's inflexible and difficult to maintain those system.s I think Bitcoin Core inherently requires a generalist knowledge of Bitcoin to really understand what parts to tweak and hwich parts to not mess with, and I think this makes it difficult for companies to collaborate.

A: From a business perspective, there's really no incentive or expectation for people to do that. I think attitudes could be changed there. Even if people can't contribute developer resources, perhaps they could contribute financially to drive forward not just Bitcoin Core but the broader bitcoin effort. Without a driving force to lead this, I think it's challenging. Core has been well-organized, but the other parts of the ecosystem are a lot less organized.